Big Data & Machine Learning: Reflections on Trust, Trustworthiness & Responsibility 1.pdf ·...

Big Data & Machine Learning:

Reflections on Trust, Trustworthiness &

Responsibility The Human Use of Machine Learning: An Interdisciplinary Workshop

Venice | 16 December 2016

Judith Simon

Associate Professor for Philosophy of Science and Technology

Technologies in Practice | IT University Copenhagen | Denmark

[email protected]

Big Data & Machine Learning:

Reflections on Trust, Trustworthiness & Responsibility

The Human Use of Machine Learning: An Interdisciplinary Workshop

Venice | 16 December 2016

Judith Simon

Associate Professor for Philosophy of Science and Technology

Technologies in Practice | IT University Copenhagen | Denmark

[email protected]

1.  Big Data: Why? How? What? Who? 2.  Big Data: Trust & Trustworthiness 3.  Conclusions

Outline

1. Big Data: Why? How? What? Who?

Big Data: Why & How?

What is the problem?

1.  Invasion of privacy? –  Illegitimate access to data versus informed consent through

payback cards?

payback cards? –  Invasion of privacy not due to the gathering of data, but due

to data processing and inferences

payback cards? –  Invasion of privacy not due to the gathering of data, but due

to data processing and inferences

à Big Data practices as epistemic practices with ethical, legal and political implications

Big Data Practices

Big Data: Why & How? E

Big Data Practices

Big Data: What?

What is big data?

Big Data: What?

Big Data: Was?

What is big data? „We define Big Data as a cultural, technological, and scholarly phenomenon that rests on the interplay of: •  1) Technology: maximizing computation power and algorithmic

accuracy to gather, analyze, link, and compare large data sets. •  2) Analysis: drawing on large data sets to identify patterns in

order to make economic, social, technical, and legal claims. •  3) Mythology: the widespread belief that large data sets offer a

higher form of intelligence and knowledge that can generate insights that were previously impossible, with the aura of truth, objectivity, and accuracy.“

(Boyd & Crawford 2012:665)

Big Data: What?

What is big data? „We define Big Data as a cultural, technological, and scholarly phenomenon that rests on the interplay of: •  1) Technology: maximizing computation power and algorithmic

accuracy to gather, analyze, link, and compare large data sets. •  2) Analysis: drawing on large data sets to identify patterns in

order to make economic, social, technical, and legal claims. •  3) Mythology: the widespread belief that large data sets offer a

higher form of intelligence and knowledge that can generate insights that were previously impossible, with the aura of truth, objectivity, and accuracy.“

(Boyd & Crawford 2012:665)

Big Data: What?

Types of (big) data

Big Data: What?

Types of (big) data •  Social media – user/usage data:

–  Explicit: comments, likes, search terms, uploads (photos, videos, ...

–  Implicit: location, views, clicks, time spent online, ...

Big Data: What?

Types of (big) data •  Social media – user/usage data:

–  Explicit: comments, likes, search terms, uploads (photos, videos, ...

–  Implicit: location, views, clicks, time spent online, ...

•  Transaction data •  Location/time-related data •  Sensor data, Internet of Things

Big Data: What?

Types of (big) data

•  Scientific data •  Not human-related data: astronomy, physics, biology, ... •  Human-related data: psychology, economy, sociology, medicine, ...

•  Medical data –  Data from clinical trials, genetic data, ... –  Data from electronic patient records, administrative hospital data, ... –  Sensor data, data from monitoring devices in clinics or care

settings, ... –  Personal health monitoring: wearable devices, home sensors, smart

phone apps, search queries and web behaviour, ...

Big Data: What?

Types of (big) data

•  Scientific data •  Not human-related data: astronomy, physics, biology, ... •  Human-related data: psychology, economy, sociology, medicine, ...

•  Medical data –  Data from clinical trials, genetic data, ... –  Data from electronic patient records, administrative hospital data, ... –  Sensor data, data from monitoring devices in clinics or care

settings, ... –  Personal health monitoring: wearable devices, home sensors, smart

phone apps, search queries and web behaviour, ...

Big Data: What?

Types of (big) data •  Citizen information/ (Open) Government Data

–  Family register: parents, birth place and date, marital status, ...

–  Address, occupation, social security number, ... –  Census data etc. –  Financial data

Big Data: What?

Data Merging

Big Data: What?

Big Data: Who?

Who collects data?

•  Industry

•  Academia

•  Governance •  (Users)

Big Data: Who?

Different functions & implications of Big Data

Big Data: Industry

New data collectors

Big Data: Industry

http://www.forbes.com/sites/davefeinleib/2012/06/19/the-big-data-landscape/

Big Data: Academia

Paradigm Shift?

Big Data for Governance

Big Data: Governance

„Big Data: Seizing Opportunities, Preserving Values“ White House Report on Big Data 2014

•  Great promises for advances in many public & private sectors –  healthcare, education, law enforcement & homeland

security, ...

•  Threats for values & civil rights –  privacy, freedom of speech and association, equality,

autonomy, ...

„Big Data: Seizing Opportunities, Preserving Values“ White House Report on Big Data 2014

•  Great promises for advances in many public & private sectors –  healthcare, education, law enforcement & homeland

security, ...

•  Threats for values & civil rights –  privacy, freedom of speech and association, equality,

autonomy, ...

2. Big Data: Trust and Trustworthiness

•  Trust in Big Data as something frequently strived for •  Yet: whom and what should we trust? •  Is this trust justified and how could we know? •  Relationship between trust, trustworthiness and

responsibility

Big Data & Trust

•  Trust in Big Data –  Trust in Big Data practices in industry, academy,

governance? –  Trust in results of data analytics? –  Trust in data, algorithms, platforms? –  Trust in the application of Big Data analytics for different

purposes? –  Trust in the actors involved in big data analytics?

•  Individual human actors: the programmer, the analysist, the sales person, the governor,...

•  Colleactive Actors: companies, governments, administrations, science,...

•  Non-human actors: self-learning algorithms as non-human actors?

Big Data & Trust

•  Trust in Big Data –  Trust in data, algorithms, platforms? –  Trust in results of data analytics? –  Trust in the application of Big Data analytics for different

purposes? –  Trust in Big Data practices in industry, academy,

governance? –  Trust in the actors involved in big data analytics?

•  Collective Actors: companies, governments, administrations, science,...

Big Data & Trust

•  Trust in Big Data as something frequently strived for •  Yet: trust is risky...

Big Data & Trust

„First of all there has to be some cause for displaying trust. There has to be defined some situation in which the person trusting is dependent on his partner; otherwise the problem does not arise. His behaviour must then commit him to this situation and make him run the risk of his trust being betrayed. In other words he must invest in what we called earlier a ‘risky investment.’ One fundamental condition is that it must be possible for the partner to abuse the trust.“ (Luhmann 1979)

Trust is “inherently subject to the risk that the other will abuse the power of discretion“ (Hardin 1993)

Big Data & Trust

„First of all there has to be some cause for displaying trust. There has to be defined some situation in which the person trusting is dependent on his partner; otherwise the problem does not arise. His behaviour must then commit him to this situation and make him run the risk of his trust being betrayed. In other words he must invest in what we called earlier a ‘risky investment.’ One fundamental condition is that it must be possible for the partner to abuse the trust.“ (Luhmann 1979)

Trust is “inherently subject to the risk that the other will abuse the power of discretion“ (Hardin 1993)

Big Data & Trust

•  Trust in Big Data as something frequently strived for •  Yet: trust is risky... •  So whom and what should we trust?

à Trust the trustworthy – and only them

Big Data & Trust

•  Trust in Big Data as complex socio-technical phenomenon

•  Artefacts mediating trust relations between humans and being a patient of trust themselves

•  Human & non-human, individual and collective actors can serve as agents and patients of trust – or can they?

Big Data & Trust

•  Trust in Big Data as complex socio-technical phenomenon –  Technologies mediating trust relations between humans

and being a patient of trust themselves –  Especially in the case of ML this trust increasingly equals

blind trust

•  Is this trust justified and how could we know? •  Relationship between trust, trustworthiness and

responsibility

Big Data & Trust

blind trust

•  Is this trust justified and how could we know? •  Relationship between trust, trustworthiness and

responsibility

Big Data & Trust

blind trust

•  Are there alternatives to such blind trust? –  To what extent can trust be justified and how can we

rationally place or withhold trust?

Big Data & Trust

blind trust

•  Are there alternatives to such blind trust? –  To what extent can trust be justified and how can we

rationally place or withhold trust?

•  Goal: trust those who are trustworthy and act responsibly

Big Data & Trust

Big Data & Trust in Academia

•  Trust as a fundamental ingredient of knowledge practices e degree based upon trust in what other people have told us, i.e. their testimony.

–  Without trusting other people, we would neither know the most basic facts of our live, such as when and where we are born, nor would we have achieved advanced scientific knowledge.

•  Moreover, to know we not only trust people, but also other entities, such as institutions, procedures or technologies

•  Trust as a fundamental ingredient of knowledge practices

–  Without trusting other people, we would neither know the most basic facts of our live nor would we have achieved advanced scientific knowledge.

–  Moreover, to know we not only trust people, but also other entities, such as institutions, procedures or technologies

–  Trust is not only placed in people, but also other entities, such as institutions, procedures or technologies

MacKenzie (2001): Mechanizing proof: computing, risk, and trust

–  Most aspects of our private and social life depend on

computing – but how can we know that computing is trustworthy?

–  MacKenzie compares human mathematical proof with formal, automated proof – and concludes:

„Yet the human community is now not the only ‚trustworthy agent‘ to which to turn: it has been joined by the machine. […] Modernity‘s‚ trust in numbers‘ can, it appears, lead back to a grounding not in trust in people, but trust in machines.“ (Mackenzie 2001:12)

Humphreys (2004): Extending Ourselves: Computational Science, Empiricism, and Scientific Method

•  Scientific knowledge is not limited to what human senses can

provide: senses can be augmented by instruments and new forms of mathematics, such as simulations.

•  Many calculations in physics are too complex to be conducted by humans alone, they depend on computers.

•  Problem of "epistemic opacity“: e.g. when a computational process is too fast for humans to follow in detail, or when there is no explicit algorithm linking inputs to outputs.

à Where do we have to start trusting in the use of simulations and other complex mathematical methods?

•  Where do we have to start trusting in the use of big data analytics and machine learning in research? –  Functional opacity: lack of access –  Epistemic opacity: lack of understanding/understandability

•  What are the alternatives?

•  Rational trust instead of blind trust? How can we signal & detect trustworthiness? –  How to avoid blind trust? –  To what extent can machine reasoning be made human-

understandable? –  By which means? Visualizations? Bias-Detection Software?

•  To what

•  Rational trust instead of blind trust? How can we signal & detect trustworthiness? –  To what extent can machine reasoning be made human-

understandable? –  By which means? Visualizations? Bias-Detection Software?

•  To what

Big Data & Trust in Industry

•  Pervasiveness of data analytics in many private sectors

•  Do we trust in data analytics? –  Vulnerability, risk, uncertainty, opacity – but choice?

•  Assessing the trustworthiness of data analytics –  Epistemological: quality of training data, algorithms,

predictions –  Ethical: discrimination, privacy, freedom, autonomy…

•  Economic context –  Data as currency –  Monopolies and distorted competition –  Governance of big data markets & alternative business

models

•  Do we trust in commercial data analytics? –  Vulnerability, risk, uncertainty, opacity – but choice?

models

predictions –  Ethical: discrimination, privacy, freedom, autonomy… –  Economic framing: data as currency; distorted

competition, ...

Big Data & Trust in Governance

Statistics as a tool for governance has a long history, hence big data is merely a continuation of this „trust in numbers“ (Porter 1995).

The term "statistics“ has been used to describe the systematic collection of demographic and economic data by the state since the 18th century „As the etymology of the word shows, statistics is connected with the construction of the state, with its unification and administration.“

(Desrosières 1998:8)

„The need to know a nation in order to govern it led to the organization of official bureaus of statistics (...).“ (Desrosières 1998:16)

à  History of states as history of statistics, of data gathering and processing

à  Knowledge <> Power

•  Big data as a trusted tool for evidence-based policy making

•  Yet to be trusted, it must demonstrate trustworthiness –  Epistemic dimension: Quality of knowledge claims based upon

big data analyses •  Difficult due to problems of access & competence

–  Ethical dimension: Privacy, discrimination, equality, autonomy, etc.

•  Even if trustworthiness was achieved, utilization still controversial •  Political dimension: functionality of knowledge in political

context, contested role of expert knowledge/scientific advice in decision making, problems of technocracy, etc.

Big Data, Trust & Governance

big data analyses •  Difficult due to problems of access & competence

big data analyses •  Difficult due to functional and epistemic opacity

•  Even if trustworthiness was achieved, utilization still controversial •  Political dimension: instrumental role of knowledge in political

context, contested relevance/role of expert knowledge/scientific advice in decision making, problems of technocracy, etc.

Conclusions

•  For BD/ML, trust is often strived for, yet trustwortiness should rather be of central concern

•  Barriers to trustworthiness are manifold –  Functional and epistemic opacity, especially regarding ML –  Economic framing incentivizes untrustworthy practices –  Political (mis)use of opaque statistical reasoning

•  How to support trustworthy data practices & enable their epistemic, ethical & political assessment?

Conclusions

•  Barriers to trustworthiness are manifold –  Functional and epistemic opacity, especially regarding ML –  Economic framing incentivizes untrustworthy practices –  (Mis)use of opaque statistical reasoning for political action

Conclusions

Hard Law Soft Law Governance By Design Education

Governance for BD/ML

Thank you for your attention!

Judith Simon

Big Data & Trust: Defining Trust

•  Good will

“(T)rusting can be betrayed, or at least let down, and not just disappointed” (Baier 1986, 235).

Trust – betrayal versus reliance – disappointment “Consider that one can rely on inanimate objects, such as alarm clocks; but when they break, one is not betrayed, although one may be disappointed. Reliance without the possibility of betrayal is not trust.” (MacLeod 2015)

Big Data & Trust – Trustworthiness – Reliance – Reliability

Big Data & Machine Learning: Reflections on Trust, Trustworthiness & Responsibility 1.pdf ·...

Documents

Transcript of Big Data & Machine Learning: Reflections on Trust, Trustworthiness & Responsibility 1.pdf ·...