intimacy & interfacerobert m ochshornalex’s apartment in brusselsmay 22, 2016

I’ve just returned from a week-and-a-half in Palestine and I have to admit my head is in a very different place from when this talk was written. The reason I’m here tonight is that the development of these ideas included a couple of Skype calls with Alex; the talk was much improved by the unlikely resonance I always seem to find with her; the unexpected coincidence of our thoughts. But coming back from Ramallah—coming back very sunburned from Jaffa—my work seems so foreign to me that I might as well be in the audience. I will read from the script, but know that it’s the wrong script for us tonight.

It’s nice to see several familiar faces in the audience. I know Marc from Jan van Eyck, which is where “my” work started, I know Alex from Solitude, which is perhaps where the crispness of my narrative began to dissolve, and I know a few others from the Constant sphere around Brussels. Some of you will need to endure a little repetition, but I hope there will be at least something new for everyone.

But before I say much more, I’d like to show a three-minute video that I made for a documentary exhibition in Copenhagen last November.

A Little More Stable

Here’s the description that accompanied the piece:

Much like the stabilized video from military drones, which allow commanders to observe and interfere with unfamiliar territory from aerial vantage, A Little More Stable takes the viewer above and outside the normal passage of motion picture time. The source material, a promotion for video stabilization software that has been commercialized from its military applications, depicts the stabilization of home movies and implies a similar steadying influence on the lives of would-be customers.

One of the great pleasures of making a piece like this is that it stretches my perceptual abilities. I spend a lot of time listening, looking, and learning from the intermediate results. I become intimate with structures and cadences between, amidst, and around the video. In some ways, I am teaching myself to “read” a language as I develop it. Unfortunately, that leaves me unable to anticipate the legibility (or illegibility) of a piece like this to those who haven’t spent time with the language. The underlying rules—this piece is the deterministic result of a computer program that I wrote—may not be apparent to you. I’ll walk through the rules:

On the top is a listing of all of the words that were uttered during the advertisement. The words are played in alphabetical order, underlined in white when they are spoken. If a word is spoken multiple times, all of the instances are played at once, on top of each other as a chorus. After each dictionary word, the successive words play at diminishing volume until we reach a comma or a period. The playback of those secondary and tertiary words is indicated by the flickering black underlines.

Underneath the word-index are animated video timelines. Each line represents 30 seconds of video. There are five lines, adding up to a total of two minutes and 30 seconds. Within each line, time is linearly distributed from left to right; the position of the left edge of the frame corresponds exactly to the timecode of the image within. If you watch the frame between the red lines, you will watch the commercial in its original form. It is moving through space at exactly the rate of cinema-time. The transcript from this red-framed video is displayed within the frame, and its audio is mixed in at a low volume. When words from the alphabetical ordering are played, the text is visualized in its position on the timeline.

All of the material in my rendering was contained within a promotional video that the nvidia corporation posted to their YouTube channel.

Despite lacking buttons or forms or other sensors, I consider this to be an interactive piece because it sits waiting for the viewer to ask it a question; it is a video in some sense, but doesn’t move through time with a single, predetermined focus.

But I also make things that are more traditionally interactive, interfaces where you can click on things, algorithms to support them. In an unexpected “return” to my first media interface, a couple of months ago I released an open source Forced Aligner (called “gentle”) to align audio files with their transcripts:

Stable Transcript Player

This is the transcript to source material from my piece.

Now I can click on any word as an index into the video, to focus my questions.

- Why is the US bombing foreign countries with “high-tech unmanned aerial vehicles that patrol the sky”?

- How can an image be used do distinguish “the bad guys” from “friendlies”?

- How much of our consumer technology has moved “from the battlefield to the playground”?

What brutality does technology carry with it from the battlefield?

• you’re going for the high danger right away. That's good dude

• a much sharper image that can last the test of time and you can torture your child with for the rest of his or her life

• From the battlefield to the playground, it's technology to make your life a little more stable

What is stability, and do we want it? If stability and explosives can exist in the same sentence, does that mean that the endgame of stability is always death?

My hope, in deconstructing this advertising video, was to raise a few of these questions. I believed, or at least hoped, that the development of tools for seeing could also help us come to the answers.

But this piece is also, intensely, self-critical. It is a juncture in my work, and so I use it as the starting point in the conversation I want to have today about intimacy in interface. It is a self-critical piece—I am equating the video timelines that I spent years dreaming up and developing and implementing with the sensation of being in a drone above a foreign territory. It’s self-critical because, while the process is for me very intimate—the years of research underlying this piece have brought me to a perhaps unique familiarity and closeness with corners of image, sound, & interaction—I have only the scantest evidence that the result of this process engenders the same intimacy in others, and in fact probably leads just as often to the opposite.

I should start with the term.

What does “intimacy” mean, if not “closeness”? What can anything mean, exactly? As far as meaning is concerned, perhaps “exactness” is impossible and the best we can do is to get close.

word2vec started as a project at Google a few years back. it is a software program that takes a large quantity of written text, and from words’ proximity to one another transforms each word into a “vector”—a list of numbers—that are then numerically comparable.

word2vec

google’s user interface to the project is no big surprise to users of their search engine. you start with a prompt, enter a word, and receive a list of results.

news & stack overflow

• abstraction• intimacy• drone

augmented augmented human intellect

I started thinking about the word “intimacy” in relation to my work through correspondence with Michael Nielsen and Joe Edelman. Michael, a quantum physicist who’s been interested in scientific interface and recently hosted a “tools for thought” event, sent us an interview with mathematician Alain Connes, where he said, in reference to a very hairy mathematical situation:

We first went back to check again these terms and realized after a while that the reason why their signs were wrong was quite subtle : we had made a conceptual mistake in doing the computation and had forgotten some crucial terms in the differential operators below the sub-principal symbol! At that point I convinced myself that we could never find this subtle correction without the intimate knowledge of all the terms of the formula which only the slow computation “by hand” could give us…(emph mine)

I wrote:

I believe that intimacy is vital to all manner of human endeavors, and that a crucial and deficient element of human activity in the 21st century is to shape our intimacies, and, paradoxically, by doing so to shape ourselves. Choosing, as Feynman (against the abacus) and Connes do, to cultivate an intimacy with numbers means acknowledging that “there is more than just 0 and 1”—that the principles alone will not suffice, and that the details, the specific, hold all meaning.

Joe encouraged me to develop the concept of intimacy in interface further, and I gave a rough 15-minute first pass at Michael’s event, for designers and scientists. Last month I expanded it into a rough 45-minute version at Brown University, and it grew further into two hours at MIT and UC Davis. I don’t want to speak for nearly that long, and we’re all lounged about on soft things. I hope this can turn into a conversation. That’s an invitation to interrupt me at any point.

A quick disclaimer, but to be clear: I don’t believe that a tool, an artwork, or an architectural environment can “create” intimacy in others, as such (phrases like “virtual reality is the ultimate empathy machine” make me sick), but I may speak loosely with the assumption that environments and techniques can be better or worse for intimacy.

Russell book

I was in falling in love when I started thinking about my work in relation to intimacy, and I can’t help to wonder how that seeped into all corners of my work and life. Here’s the Russell bootleg I made her.

codex @ codex (?!)

I wonder if “literary hackathon” sounds like an oxymoron to you.

A moment from the demos that stuck with me was the presentation of “Kindlr.” Their idea was to make a dating app based on literary holdings and interests. users looking for love could be effortlessly onboarded into totally-not-awkward dates with plenty to talk about. If all else failed, you could read together. quote: “In bed.” This was an awkward moment. The front-end engineer slipped the line out apologetically, and it was painful for the whole room, but I felt there might be a germ of some truth lurking there. Books are intimate objects, solitary or romantic, and what pains the public schools must go through to dull them into pedagogy. I think part of the intimacy of books comes from their stillness, which curriculum disrupt by rushing the pages through time, and which general purpose computing tablets dilute with the dull of batteries and the pull of push notifications.

Destroyed Russell book!

Out of love, back into a state of confusion. I had a Skype call with my friend Alex about this notion of intimacy, and she thought that confusion must play a role. A new project, for her, is entry into a pitch black room. Shouting out to catch the echoes and a guess at the size. Grasping

around. This talk will aspire to some confusion for all of us.

Reading is an important reference point for my work.

Americanah Cards

alt snippet UI.“closeness” in the physical world… not

subject to any computation

Hyperopia Thing

the “reading is writing” joke of receipts turning on me

Journalism in the Desert of the Real

how little can be “stored” in the computer?

I’ll come back to physical, non-screen-based

interactions shortly. I want to back up a little bit and talk about computing paradigms.

against abstraction

Abstraction is a technique to help us forget.

There are many reasons we would want to forget a good many things.

We use terms like “generalization,” “higher level,” “don’t repeat yourself,” “encapsulation,” “business logic,” and “eliminating boilerplate” when what we mean is that we don’t want to concern ourselves with the intimate details of a task at hand.

This would imply that we think of our lives as a dependency graph of tasks to be completed, but I would claim that creative and intellectual work is closer to speaking a language than processing a workflow. Nothing is stable, definitions are circular, every utterance rewrites its every word, insight is a simultaneity of meaning, and profundity comes through the temporal sequence of process rather than the compilation and execution of result.

Generalization is the loss of the individual to the collective, the instance to the class, the particular to the composite. Statistics and surveillance are together entwined—from Sir Francis Galton’s photographic composites to the MIT Media Lab’s Eigenfaces—in an aesthetic blurring of sublime truth and raw bigotry.

Externalization is a method to replace knowing with seeing, a methodology more amenable to oversight than insight.

Consider the hollowness of learning the periodic table without having a deep relationship to its elements. When connected to the depth of particulars, Mendeleev’s table can be understood as a poem, as Primo Levi puts it. But without the concrete, it becomes a smokescreen, a barrier to insight, a dangerous surrogate.

• • •

On surrogates:

ZKM

control and illusionabstraction/quantitative, temporal/qualitative

Livezoom

time -> space

Hilbert

time -> space, non-text-inspired

Metric Feedback

text -> column -> textual feedback

App/territory

piano is not always the best spatial representation of the notes it will produce

SFX

audio -> t-SNE, related to some collaborations w/composers

The literal reading of “closeness” mapped to “distance” is limiting.Similarity need not be so simple,

Montage Interdit

Juxtaposition as a form of closenessAuthored “closeness”meaning in the viewer’s head

…the project was interesting for a very different reason than I anticipated

I was coming from a research into Compression, which I can go into if necessary […]

Compression -- as pertains to the moving image -- is generally understood on a technical level. Movies are "compressed" into smaller file sizes to allow for their distribution through digital mediums, most recently the Internet. Compression, then, is the basis and enabling force behind digital networked communication. When it is successful, compression is invisible, imperceptible to the human eye; a bias of the flow and intensity of imagery towards recognition by the human visual system. Compression inherently involves a determination of what information is most important, and a purge of the rest. For example, compressed images lose disproportionately

more detail in their dark regions, where human eyes are less sensitive.

Sublimation

Considered as a filtering protocol, the idea of compression can be applied more broadly to the production of moving images: editing a film entails a "compression" of many hours of footage into a comparatively short piece. In many cases, this compression too is deemed successful when it is imperceptible. The final result of editing should appear "complete"

even though it is obviously missing most of what was filmed (not to mention what was never even recorded in the first place). And how few people there are who determine the parameters of compression -- the relative importance of information -- for so many! Compression of information into video is highly centralized.

GOD

Directly between the social and technical compression of the moving picture is the editing interface: it is designed to facilitate the social process of compression and then perform an additional level of algorithmic compression to finally yield a form suitable for dissemination. And unsurprisingly its structure mirrors and re-enforces the social structure in which it operates: it is designed for use by a single operator to determine a canonical subset of closely-guarded, privately-held, source video for "read-only" distribution to a passive audience.

interlace.videovortex

It is at this site of production, the video editing interface, where I have started a multifaceted investigation into "compression," in particular as concerns the organization and distribution of media.

This investigation failed.

Video Binders

embarrassing for my video timelines

Blue Book Silver Book

“digital archive” installation doesn’t need to be between screens

I have no conclusion; the work is ongoing.- or -

stabs at intimacy

- intimate interfaces cannot be “wireframed” or otherwise faked. they are a lens more than a frame.

- repetition can lead to new discoveries. use is not “monotonous” and therefore automation is never the goal.

- meaning “passes through” the system. “the answer” is never directly modeled, but occurs in the heads of humans.

- programming intimate interfaces is more intimate than using them. and since this intimacy is fundamentally lingual, it is a non-intuitive process poorly served by “end user” tooling.

conclusion

Search, and re-search, must be processes of acquiring intimacy not abstracting to an outline or de-skilling to a workflow.