ALGORITHMIC FAIRNESS: MEASURES, METHODS AND …suresh/static/files/tutorial.pdfexample for hiring)...
Transcript of ALGORITHMIC FAIRNESS: MEASURES, METHODS AND …suresh/static/files/tutorial.pdfexample for hiring)...
ALGORITHMIC FAIRNESS: MEASURES, METHODS AND REPRESENTATIONS
SURESH VENKATASUBRAMANIAN UNIVERSITY OF UTAH
PODS 2019
RIPPED FROM THE HEADLINES…
RIPPED FROM THE HEADLINES…
RIPPED FROM THE HEADLINES…
RIPPED FROM THE HEADLINES…
FAT* RESEARCH AREAS.
Computer Science
Machine Learning
Algorithms
Databases
HCI
Other
SociologyThe LawEconomicsPolitical SciencePhilosophyMedia Studies
GOALS FOR THIS TUTORIAL
GOALS FOR THIS TUTORIAL
• An overview of the state of play in (some) areas of research in fairness
GOALS FOR THIS TUTORIAL
• An overview of the state of play in (some) areas of research in fairness
• Some open questions coming out of these areas
GOALS FOR THIS TUTORIAL
• An overview of the state of play in (some) areas of research in fairness
• Some open questions coming out of these areas
• New directions and challenges: centering the affected and introducing context.
GOALS FOR THIS TUTORIAL
• An overview of the state of play in (some) areas of research in fairness
• Some open questions coming out of these areas
• New directions and challenges: centering the affected and introducing context.
• Overarching concern: thinking about the larger context is crucial if we want to formalize and interpret fairness without making huge mistakes
GOALS FOR THIS TUTORIAL
• An overview of the state of play in (some) areas of research in fairness
• Some open questions coming out of these areas
• New directions and challenges: centering the affected and introducing context.
• Overarching concern: thinking about the larger context is crucial if we want to formalize and interpret fairness without making huge mistakes
DISCLAIMER
This is my idiosyncratic view of the
field
MEASURES
DEFINING (UN)FAIRNESS
DEFINING (UN)FAIRNESS
(a1, …, ak)
p(a1,…,ak)(x)
Think of a binary classification task
DEFINING (UN)FAIRNESS
Fairness can be expressed as a function of the classifier , the protected attribute , the training labels
Φf
pyi
Given where and the goal is to find such that
is minimized.
(x1, y1), (x2, y2), …, (xn, yn) ∈ X × Y x = (z, p)f ∈ ℱ
∑ ℓ( f(xi), yi)
protected
unprotected
MEASURES OF FAIRNESS
Individual fairness:
Demographic parity (and friends):
Equalized error rates:
(group sensitive) Calibration:
D( f(x), f(x′�)) ≤ d(x, x′�)
Pr[ f(x) = 1 |p = 1] ≈ Pr[ f(x) = 1 |p = 0]
Pr[ f(x) ≠ y |p = 1] ≈ Pr[ f(x) ≠ y |p = 0]
∀g, r, Pr[y = 1 ∣ f(x) = r, p = g] = r
MEASURES OF FAIRNESS
Individual fairness:
Demographic parity (and friends):
Equalized error rates:
(group sensitive) Calibration:
D( f(x), f(x′�)) ≤ d(x, x′�)
Pr[ f(x) = 1 |p = 1] ≈ Pr[ f(x) = 1 |p = 0]
Pr[ f(x) ≠ y |p = 1] ≈ Pr[ f(x) ≠ y |p = 0]
∀g, r, Pr[y = 1 ∣ f(x) = r, p = g] = r
Fairness can be expressed as a function of the classifier , the protected attribute , the training labels
Φf
pyi
individual fairnessdemographic parity
equalized error rates
CONDITIONAL PARITY [RSZ17]
ℒ(x ∣ a = a, z = z) = ℒ(x ∣ a = a′�, z = z)∀a, a′�
Examples: • = predicted probability of positive outcome, = group, =
Demographic parity [RP07] • = decision, = group, = true outcome
equalized odds [HPS16] • x = predicted probability of positive outcome, a = group, z = actual
probability of outcome Group-sensitive calibration.
x a z ∅
x a z
NORMATIVE DIMENSIONS TO FAIRNESS
Normative: establishing, standardizing or pertaining to a norm.
All measures of fairness carry normative positioning within them
NORMATIVE DIMENSIONS TO FAIRNESS
Normative: establishing, standardizing or pertaining to a norm.
All measures of fairness carry normative positioning within them
• Individual fairness: there exists an objective measure of ability and fairness is making sure people of similar ability are treated similarly
NORMATIVE DIMENSIONS TO FAIRNESS
Normative: establishing, standardizing or pertaining to a norm.
All measures of fairness carry normative positioning within them
• Individual fairness: there exists an objective measure of ability and fairness is making sure people of similar ability are treated similarly
• Demographic parity: Group identity should have nothing to do with selection for a task.
NORMATIVE DIMENSIONS TO FAIRNESS
Normative: establishing, standardizing or pertaining to a norm.
All measures of fairness carry normative positioning within them
• Individual fairness: there exists an objective measure of ability and fairness is making sure people of similar ability are treated similarly
• Demographic parity: Group identity should have nothing to do with selection for a task.
• Equalized odds: Groups may have different innate skill levels, but we should make mistakes equally.
MORE HIDDEN ASSUMPTIONS
MORE HIDDEN ASSUMPTIONS
• All groups are equivalent and unfair treatment of one is the same as unfair treatment of another.
MORE HIDDEN ASSUMPTIONS
• All groups are equivalent and unfair treatment of one is the same as unfair treatment of another.
• All instances of unfairness boil down to individual decisions about people, and not structural factors that create the data used for learning bias decisions [FBG19]
MORE HIDDEN ASSUMPTIONS
• All groups are equivalent and unfair treatment of one is the same as unfair treatment of another.
• All instances of unfairness boil down to individual decisions about people, and not structural factors that create the data used for learning bias decisions [FBG19]
• All instances of unfairness come from the process of making decisions. [OKBTG18]
MORE HIDDEN ASSUMPTIONS
• All groups are equivalent and unfair treatment of one is the same as unfair treatment of another.
• All instances of unfairness boil down to individual decisions about people, and not structural factors that create the data used for learning bias decisions [FBG19]
• All instances of unfairness come from the process of making decisions. [OKBTG18]
Need a way to create “context knobs” for the design of fairness measures.
EXTENSIONS: REGRESSION
Given , find a mapping such that
is minimized.
• What is an appropriate form of fairness?
• What is the context for this? (credit score assignment?)
• What would the right normative concerns be? (POTS: [OKBTG18])
(xi, yi), xi ∈ ℝd, yi ∈ ℝ f : ℝd → ℝ
∑i
∥f(xi) − yi∥2
EXTENSIONS: RANKING
• Build a ranking scheme to rank individuals from top to bottom (for example for hiring)
• What are forms of fairness in rankings [YS17,ZBCHMB-Y17,CSV17,AJSD19,SJ]
• “demographic parity”: proportion of people from different groups in each top-k section should be roughly the same
• “quotas”: each group should have an upper/lower bound on number of members in top-k for different k
• generalize rank to “exposure” and equalize it [AS17,19,BGW18]
EXTENSIONS: CLUSTERING
• Given points labeled red and blue, find a good clustering so that each cluster has the same proportion of groups as in the overall population. [CKLV18]
EXTENSIONS: CLUSTERING
• Given points labeled red and blue, find a good clustering so that each cluster has the same proportion of groups as in the overall population. [CKLV18]
EXTENSIONS: CLUSTERING
• Given points labeled red and blue, find a good clustering so that each cluster has the same proportion of groups as in the overall population. [CKLV18]
• Good idea: each cluster should be representative
EXTENSIONS: CLUSTERING
• Given points labeled red and blue, find a good clustering so that each cluster has the same proportion of groups as in the overall population. [CKLV18]
• Good idea: each cluster should be representative
• Bad idea: if we consider this an example of redistricting, then the minority party loses all the seats!
METHODS
FAIRNESS IN CLASSIFICATION
Modify the training data prior to the
training process
Add fairness constraints
when learning the model
Modify the labels after
training
MODIFYING THE TRAINING DATA
• PRO: Might be able to account for data bias, deal with black box classifier
• CON: Might need to modify data extensively to remove skew - this is not well defined.
• CON: No transparency for what concerns are being addressed by skew elimination. Other biases might carry through.
TRAINING WITH A REGULARIZER
• PRO: Flexibility to work with any data. More control over learned models
• CON: All regularizers are proxies - unintended consequences.
• CON: Even harder to explain outcomes.
POST-PROCESSING THE PREDICTIONS
• PROS: Works with black box classifier and any training data. Has certain optimality properties.
• CON: Might very well be illegal (in the US, in certain sectors).
• CON: What is principled argument for post-processing labels?
MODIFYING THE TRAINING DATA
Given X, construct such that
• (we cannot predict p)
• A predictor learned on is similar to a predictor learned on .
Notes:
• Should try to change X’ minimally.
• Other biases might remain in X’.
X′� = g(X)
X′� ⊥ p(X)
f′� X′� fX
TRAINING WITH A REGULARIZER
Translate fairness conditions into constraints encoded into the optimization.
Demographic parity [ZVRG17a, ZVRG17b]:
In general constraints might not be convex, and so proxies are needed.
|∑p(x)=1 h(x)
N1−
∑p(x)=0 h(x)
N0| ≤ ϵ
POST-PROCESSING LABELS
POST-PROCESSING LABELS
• Run the training process without any intervention and construct a derived predictor based on the (joint distribution of) learned model, group attributes and ground truth outcome.
POST-PROCESSING LABELS
• Run the training process without any intervention and construct a derived predictor based on the (joint distribution of) learned model, group attributes and ground truth outcome.
POST-PROCESSING LABELS
• Run the training process without any intervention and construct a derived predictor based on the (joint distribution of) learned model, group attributes and ground truth outcome.
• At prediction time only use information from learned model and group outcome. [HPS16]
x = (z, p)
DESIGN DIMENSIONS
FEEDBACK FROM MODELUpdate mode
(a1, …, ak)
p(a1,…,ak)(x, t)
Batch learning with feedback
Model output contaminates training data Training data no longer drawn from “true” distribution.
[EFNSV18a,EFNSV18b]
DEALING WITH FEEDBACK
• What the system learns depends mostly on initial conditions, not the actual data.
• Small differences in input probabilities lead to huge differences in output predictions.
To Predict And Serve [LI16]
[EFNSV18a,EFNSV18b]
DEALING WITH FEEDBACK
• What the system learns depends mostly on initial conditions, not the actual data.
• Small differences in input probabilities lead to huge differences in output predictions.
To Predict And Serve [LI16]
Need to use reinforcement
learning rather than supervised
learning, but no one does that
FEEDBACK FROM BEHAVIOR
• Police presence might change the behavior of residents
• this is a good thing! BUT will render model inaccurate
• If a model is trained on one distribution, it will in general not work if underlying distribution changes
• Monitoring for changes in underlying distribution is hard.
MORE FEEDBACK PROBLEMS
• Strategic classification: What happens if players try to “game” the model?
MORE FEEDBACK PROBLEMS
• Strategic classification: What happens if players try to “game” the model?
• BAD: I realize that getting more credit cards will increase my credit score, so I go out and get some.
MORE FEEDBACK PROBLEMS
• Strategic classification: What happens if players try to “game” the model?
• BAD: I realize that getting more credit cards will increase my credit score, so I go out and get some.
• GOOD: I realize that signing up for a regular health checkup will reduce my insurance costs, so I do it.
MORE FEEDBACK PROBLEMS
• Strategic classification: What happens if players try to “game” the model?
• BAD: I realize that getting more credit cards will increase my credit score, so I go out and get some.
• GOOD: I realize that signing up for a regular health checkup will reduce my insurance costs, so I do it.
What's the difference?
PIPELINES
• Decisions are made in multiple stages.
• We want fairness guarantees to compose.
• BAD: In most settings, fairness guarantees do NOT compose
• INTERESTING: Under certain modeling assumptions, it’s better to intervene earlier in the pipeline rather than later.
Under what conditions can we compose fairness guarantees and how does this guide interventions?
College admission
JobGraduate school
PIPELINES
• Decisions are made in multiple stages.
• We want fairness guarantees to compose.
• BAD: In most settings, fairness guarantees do NOT compose
• INTERESTING: Under certain modeling assumptions, it’s better to intervene earlier in the pipeline rather than later.
Under what conditions can we compose fairness guarantees and how does this guide interventions?
College admission
JobGraduate school
STABILITY I: ALGORITHMS VARY
• Current approaches to achieving fairness have very different operating characteristics. [FSVCHR19]
STABILITY 2: WHAT CAN WE DO
• Stable and fair classification [HV19]
in a recent study, Friedler et al. observed that fair classification algorithms may not be stable with respect to variations in the training dataset -- a crucial consideration in several real-world applications. Motivated by their work, we
study the problem of designing classification algorithms that are both fair and stable. We propose an extended framework based on fair classification
algorithms that are formulated as optimization problems, by introducing a stability-focused regularization term.
Can we show that fairness guarantees generalize?
FAIRNESS AND PRIVACY
FAIRNESS AND PRIVACY
• VIEW: privacy helps with fairness because sensitive information about individuals will not be leaked
FAIRNESS AND PRIVACY
• VIEW: privacy helps with fairness because sensitive information about individuals will not be leaked
• VIEW: privacy hurts fairness because we can hide discrimination by not collecting sensitive information but inferring it.
FAIRNESS AND PRIVACY
• VIEW: privacy helps with fairness because sensitive information about individuals will not be leaked
• VIEW: privacy hurts fairness because we can hide discrimination by not collecting sensitive information but inferring it.
•
FAIRNESS AND PRIVACY
Attempts to attack private data, or operate on private data, exhibit disparate effects on different groups.
• Disparate Vulnerability: on the unfairness of privacy attacks against machine learning. [YKT19]
• Fair Decision Making using privacy-protected data. [KMPHMM19]
• Differential Privacy has disparate impact on model privacy [BS19]
How do privacy and fairness really interact?
REPRESENTATIONS
NEW REPRESENTATIONS
• Core idea: a learned representation is “good” if it conceals information about protected attribute maximally, while affecting ability to classify minimally [ZWSPD13, MCPZ18].
x = (z, p)
x′� = (z′�, p)
Cannot predict from p z′� Can predict from y z′�
DISENTANGLED REPRESENTATIONS [H+18]
• Mask signal about protected attributes but without destroying realism of data.
• Formally,
(z, p) (w, p) ( z, p)Disentangled
representation
ϕ ϕ
Input Reconstruction
and are independentw p
DISENTANGLED REPRESENTATIONS
• Disentangled representations can be used to preprocess training data (because is both realistic and independent of )
• Disentangled representations can be used to determine the influence of protected attributes on the classification [MPFSV19]
• But we still lack a principled explanation of bias in representations.
z p
BIAS IN REPRESENTATIONS
• Can we identify bias in existing (learned) representations and correct it?
BIAS IN REPRESENTATIONS
• Can we identify bias in existing (learned) representations and correct it?
man
womandoctor
nurse
BIAS IN REPRESENTATIONS
• Can we identify bias in existing (learned) representations and correct it?
man
womandoctor
nurse
BIAS IN REPRESENTATIONS
• Can we identify bias in existing (learned) representations and correct it?
man
womandoctor
nurse
man
woman
doctor nurse
THE GEOMETRY OF BIAS
• “bias” = distortion of representation along the "gender axis”
man
womandoctor nurse
THE GEOMETRY OF BIAS
• “bias” = distortion of representation along the "gender axis”
• “mitigation” = reversal of this distortion.
man
womandoctor nurse
THE GEOMETRY OF BIAS
• “bias” = distortion of representation along the "gender axis”
• “mitigation” = reversal of this distortion.
• Stay tuned for more….
man
womandoctor nurse
PRINCIPAL COMPONENT ANALYSIS
Given matrix M ∈ ℝm×n, find M ∈ ℝm×n with rank d such that
∥M − M∥F is minimized
PRINCIPAL COMPONENT ANALYSIS
Given matrix M ∈ ℝm×n, find M ∈ ℝm×n with rank d such that
∥M − M∥F is minimized
M = MWW⊤ where columns of WM⊤Mare the top eigenvectors of
[SAMADI ET AL, 2018]
FAIR PRINCIPAL COMPONENT ANALYSIS
[SAMADI ET AL, 2018]
FAIR PRINCIPAL COMPONENT ANALYSIS
L(X, Z ) = ∥X − Z∥F − ∥X − X∥F
[SAMADI ET AL, 2018]
FAIR PRINCIPAL COMPONENT ANALYSIS
L(X, Z ) = ∥X − Z∥F − ∥X − X∥F
A
B
minU
max L(A, UA), L(B, UB) for rank-d U ∈ ℝm×n
[SAMADI ET AL, 2018]
FAIR PRINCIPAL COMPONENT ANALYSIS
L(X, Z ) = ∥X − Z∥F − ∥X − X∥F
A
B
minU
max L(A, UA), L(B, UB) for rank-d U ∈ ℝm×n
Thm: Can obtain optimal U with rank at most d+1 (solve an SDP followed by LP)
WHAT NEXT?
CENTERING THE SOLUTION
CENTERING THE HARMED
CENTERING THE HARMS
…returning to the idea of unfairness suggests several new areas of inquiry, including quantifying different kinds of unfairness and bias….
Quantifying types of unfairness may not only add to the problems that machine learning can address, but also accords with realities of sentencing and policing behind much of the fairness research today: Individuals seeking justice do so when they believe that something has been unfair.
— 50 years of test (un)fairness: lessons for machine learning [HM19]
CENTERING THE HARMS
…returning to the idea of unfairness suggests several new areas of inquiry, including quantifying different kinds of unfairness and bias….
Quantifying types of unfairness may not only add to the problems that machine learning can address, but also accords with realities of sentencing and policing behind much of the fairness research today: Individuals seeking justice do so when they believe that something has been unfair.
— 50 years of test (un)fairness: lessons for machine learning [HM19]
We should think about
UNFAIRNESS rather than fairness
RECOURSE
Given positively and negatively labeled points, find a line separating them such that the margin is maximized.
RECOURSE FOR A (BAD) DECISION
Recourse: the ability of a person to change the decision of a model through actionable input variables [USL2019]
[DGNV19, ONGOING]
RECOURSE-EQUALIZED CLASSIFICATION
[DGNV19, ONGOING]
RECOURSE-EQUALIZED CLASSIFICATION
RECOURSE-EQUALIZED CLASSIFICATION
Given P = {(x1, y1), (x2, y2), …, (xn, yn)}, (xi, yi) ∈ ℝd × {+1, − 1}
min ∥w∥2
s.t. ∀i yi(w ⋅ xi + b) ≥ 1
RECOURSE-EQUALIZED CLASSIFICATION
Given P = {(x1, y1), (x2, y2), …, (xn, yn)}, (xi, yi) ∈ ℝd × {+1, − 1}
min ∥w∥2
s.t. ∀i yi(w ⋅ xi + b) ≥ 1
min ∥w∥2
s.t. ∀i yi(w ⋅ xi + b) ≥ 1|rw,b(A) − rw,b(B) | ≤ ϵ
RECOURSE-EQUALIZED CLASSIFICATION
Given P = {(x1, y1), (x2, y2), …, (xn, yn)}, (xi, yi) ∈ ℝd × {+1, − 1}
min ∥w∥2
s.t. ∀i yi(w ⋅ xi + b) ≥ 1
P = A ∪ B
rw,b(S) = ∑(x,y)∈S
w ⋅ x + b∥w∥|S− |
S− = {(x, y) ∈ S ∣ y = − 1}
min ∥w∥2
s.t. ∀i yi(w ⋅ xi + b) ≥ 1|rw,b(A) − rw,b(B) | ≤ ϵ
RECOURSE-EQUALIZED CLASSIFICATION
Given P = {(x1, y1), (x2, y2), …, (xn, yn)}, (xi, yi) ∈ ℝd × {+1, − 1}
min ∥w∥2
s.t. ∀i yi(w ⋅ xi + b) ≥ 1
P = A ∪ B
rw,b(S) = ∑(x,y)∈S
w ⋅ x + b∥w∥|S− |
S− = {(x, y) ∈ S ∣ y = − 1}
min ∥w∥2
s.t. ∀i yi(w ⋅ xi + b) ≥ 1|rw,b(A) − rw,b(B) | ≤ ϵ
GIVEN A COLLECTION OF LABELED POINTS, CAN WE COMPUTE A RECOURSE-EQUALIZED
MAXIMUM MARGIN LINEAR CLASSIFIER?
FAIRNESS IN SOCIAL NETWORKS
• Social standing [Coleman] within a network confers utility on an individual.
• Social “position” in a network is a class marker defined by the network, not the individual.
• Should we be considered about discrimination based on social position? [boyd, Marwick and Levy]
INFORMATION ACCESS
• Social networks grow through recommendations as well as organically
• Network position confers advantage ([Granovetter])
• Access to information that improves network position relies on …. network position
• “edges in social network” == “biased input data”
INFLUENCE MAXIMIZATION
Given a graph, a mechanism for spreading information and k seeds, how many nodes can be be
influenced?
INFLUENCE MAXIMIZATION
Given a graph, a mechanism for spreading information and k seeds, how many nodes can be be
influenced?
INFLUENCE MAXIMIZATION
Given a graph, a mechanism for spreading information and k seeds, how many nodes can be be
influenced?
INFLUENCE MAXIMIZATION
Given a graph, a mechanism for spreading information and k seeds, how many nodes can be be
influenced?
INFLUENCE MAXIMIZATION
Given a graph, a mechanism for spreading information and k seeds, how many nodes can be be
influenced?
INFLUENCE MAXIMIZATION
Given a graph, a mechanism for spreading information and k seeds, how many nodes can be be
influenced?
GAPS IN INFORMATION ACCESS [FBBFSV19]
• New measure of access gap in a network
• Axiomatic considerations and a proposed cost function.
• Study of how to intervene in a network (by adding edges) to improve access gaps • Theoretical (negative) results • Empirical study of heuristics.
HARMS OF REPRESENTATION
Crawford, The Trouble with Bias, NeurIPS 2017 Keynote
STEREOTYPING
“associations and beliefs about the characteristics and attributes of a group and its members that shape how people think about and
respond to the group”
— SAGE handbook of prejudice, stereotyping and discrimination.
A specific mechanism for stereotyping:
…the tendency to assign characteristics to all members of a group based on stereotypical features shared by a few…
A MODEL FOR STEREOTYPING [AFSV19]
Points regress towards an exemplar
pα = (1 − α)p + αc
CONCLUSIONS
TAKEAWAYS
TAKEAWAYS
• We are the “end of the beginning” in algorithmic fairness.
TAKEAWAYS
• We are the “end of the beginning” in algorithmic fairness.
• Our focus should always be on the broader ways in which algorithmic systems influence society.
TAKEAWAYS
• We are the “end of the beginning” in algorithmic fairness.
• Our focus should always be on the broader ways in which algorithmic systems influence society.
• We must be bold in reimagining how we can use algorithms and tech ... for good?
ACM FAT* 2020
CRAFT
CRAFT: Critiquing and Rethinking Accountability, Fairness and Transparency
A number of prominent studies acknowledge that addressing the greater societal problems due to the introduction of automation, machine learning algorithms and optimization systems may require more holistic approaches.
In the spirit of reflection and response, we are planning a call for contributions that invites academics and different communities of practice (including journalism, advocacy, organizing, education, art, public authorities) to propose workshops, panels, debates and other formats that will be co-located with ACM FAT* 2020. The details of this call will be announced shortly.
Seda Gürses, Seeta Peña Gangadharan, Suresh Venkatasubramanian
ACKNOWLEDGEMENTS
• Sorelle Friedler and Carlos Scheidegger (11 papers and counting!)
• Solon Barocas, Andrew Selbst, Karen Levy, danah boyd and Seda Gürses for perspectives on the world outside CS
• Mohsen Abbasi, Ashkan Bashardoust, Danielle Ensign, Scott Neville, Pegah Nokhiz, Chitradeep Dutta Roy - my students past and present.
• The entire FAT* community.
POST-TALK UPDATES
• References have been added.
• Note that this talk skipped a number of key topics in the area:
• fairness and causality
• methods for determining the influence of variables on outcomes (the broader area of audit mechanisms)
• the entire area of explainability/interpretability
REFERENCES
• [1] J. S. Coleman, “Social capital in the creation of human capital,” American journal of sociology, vol. 94, pp. S95–S120, 1988.
• [2] M. S. Granovetter, “The strength of weak ties,” in Social networks, Elsevier, 1977, pp. 347–367.
• [3] D. Boyd, K. Levy, and A. Marwick, “The networked nature of algorithmic discrimination,” Data and Discrimination: Collected Essays. Open Technology Institute, 2014.
• [4] D. Ensign, S. A. Friedler, S. Neville, C. Scheidegger, and S. Venkatasubramanian, “Runaway Feedback Loops in Predictive Policing,” in Conference on Fairness, Accountability and Transparency, 2018, pp. 160–171.
• [5] D. Ensign, F. Sorelle, N. Scott, S. Carlos, and V. Suresh, “Decision making with limited feedback,” in Algorithmic Learning Theory, 2018, pp. 359–367.
• [6] M. Abbasi, S. Friedler, C. Scheidegger, and S. Venkatasubramanian, “Fairness in representation: quantifying stereotyping as a representational harm,” in Proceedings of the 2019 SIAM International Conference on Data Mining, 0 vols., Society for Industrial and Applied Mathematics, 2019, pp. 801–809.
• [7] B. Fish, A. Bashardoust, D. Boyd, S. Friedler, C. Scheidegger, and S. Venkatasubramanian, “Gaps in Information Access in Social Networks?,” in The World Wide Web Conference, New York, NY, USA, 2019, pp. 480–490.
• [8] B. Hutchinson and M. Mitchell, “50 Years of Test (Un)Fairness: Lessons for Machine Learning,” in Proceedings of the Conference on Fairness, Accountability, and Transparency, New York, NY, USA, 2019, pp. 49–58.
REFERENCES
• [9] I. Higgins et al., “Towards a Definition of Disentangled Representations,” arXiv:1812.02230 [cs, stat] , Dec. 2018.
• [10] E. Bagdasaryan and V. Shmatikov, “Differential Privacy Has Disparate Impact on Model Accuracy,” arXiv:1905.12101 [cs, stat] , May 2019.
• [11] S. Kuppam, R. Mckenna, D. Pujol, M. Hay, A. Machanavajjhala, and G. Miklau, “Fair Decision Making using Privacy-Protected Data,” arXiv:1905.12744 [cs] , May 2019.
• [12] M. Yaghini, B. Kulynych, and C. Troncoso, “Disparate Vulnerability: on the Unfairness of Privacy Attacks Against Machine Learning,” arXiv:1906.00389 [cs, stat] , Jun. 2019.
• [13] L. Huang and N. Vishnoi, “Stable and Fair Classification,” in International Conference on Machine Learning, 2019, pp. 2879–2890.
• [14] A. Bower, S. N. Kitchen, L. Niss, M. J. Strauss, A. Vargas, and S. Venkatasubramanian, “Fair Pipelines,” arXiv:1707.00391 [cs, stat] , Jul. 2017.
• [15] C. Dwork and C. Ilvento, “Fairness Under Composition,” arXiv:1806.06122 [cs, stat] , Jun. 2018.
• [16] K. Lum and W. Isaac, “To predict and serve?,” Significance, vol. 13, no. 5, pp. 14–19, 2016.
REFERENCES
• [17] B. Woodworth, S. Gunasekar, M. I. Ohannessian, and N. Srebro, “Learning Non-Discriminatory Predictors,” arXiv:1702.06081 [cs] , Feb. 2017.
• [18] A. J. Biega, K. P. Gummadi, and G. Weikum, “Equity of Attention: Amortizing Individual Fairness in Rankings,” in The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, New York, NY, USA, 2018, pp. 405–414.
• [19] G. Farnadi, B. Babaki, and L. Getoor, “Fairness in Relational Domains,” in Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, New York, NY, USA, 2018, pp. 108–114.
• [20] S. Mitchell, E. Potash, S. Barocas, A. D’Amour, and K. Lum, “Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions,” arXiv:1811.07867 [stat] , Nov. 2018.
• [21] C. T. Marx, R. L. Phillips, S. A. Friedler, C. Scheidegger, and S. Venkatasubramanian, “Disentangling Influence: Using Disentangled Representations to Audit Model Predictions,” arXiv:1906.08652 [cs, stat] , Jun. 2019.
• [22] B. Ustun, A. Spangher, and Y. Liu, “Actionable Recourse in Linear Classification,” in Proceedings of the Conference on Fairness, Accountability, and Transparency, New York, NY, USA, 2019, pp. 10–19.
• [23] S. Samadi, U. Tantipongpipat, J. H. Morgenstern, M. Singh, and S. Vempala, “The Price of Fair PCA: One Extra dimension,” in Advances in Neural Information Processing Systems 31, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds. Curran Associates, Inc., 2018, pp. 10999–11010.
• [24] D. Madras, E. Creager, T. Pitassi, and R. Zemel, “Learning Adversarially Fair and Transferable Representations,” arXiv:1802.06309 [cs, stat] , Feb. 2018.
REFERENCES
• [25] R. Overdorf, B. Kulynych, E. Balsa, C. Troncoso, and S. Gürses, “POTs: Protective Optimization Technologies,” Jun. 2018.
• [26] R. Overdorf, B. Kulynych, E. Balsa, C. Troncoso, and S. Gürses, “Questioning the assumptions behind fairness solutions,” Nov. 2018.
• [27] K. Yang, J. Stoyanovich, A. Asudeh, B. Howe, H. V. Jagadish, and G. Miklau, “A Nutritional Label for Rankings,” Proceedings of the 2018 International Conference on Management of Data - SIGMOD ’18, pp. 1773–1776, 2018.
• [28] Y. Ritov, Y. Sun, and R. Zhao, “On conditional parity as a notion of non-discrimination in machine learning,” arXiv:1706.08519 [cs, stat] , Jun. 2017.
• [29] F. Chierichetti, R. Kumar, S. Lattanzi, and S. Vassilvitskii, “Fair Clustering Through Fairlets,” in Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds. Curran Associates, Inc., 2017, pp. 5029–5037.
• [30] A. Singh and T. Joachims, “Fairness of Exposure in Rankings,” arXiv:1802.07281 [cs] , Feb. 2018.
• [31] S. A. Friedler, C. Scheidegger, S. Venkatasubramanian, S. Choudhary, E. P. Hamilton, and D. Roth, “A comparative study of fairness-enhancing interventions in machine learning,” arXiv:1802.04422 [cs, stat] , Feb. 2018.
• [32] M. Zehlike, F. Bonchi, C. Castillo, S. Hajian, M. Megahed, and R. Baeza-Yates, “FA*IR: A Fair Top-k Ranking Algorithm,” arXiv:1706.06368 [cs] , pp. 1569–1578, 2017.
REFERENCES
• [33] K. Yang and J. Stoyanovich, “Measuring Fairness in Ranked Outputs,” arXiv:1610.08559 [cs] , Oct. 2016.
• [34] L. E. Celis, D. Straszak, and N. K. Vishnoi, “Ranking with Fairness Constraints,” arXiv:1704.06840 [cs] , Apr. 2017.
• [35] A. Asudehy, H. V. Jagadishy, J. Stoyanovichz, and G. Das, “Designing Fair Ranking Schemes,” arXiv:1712.09752 [cs] , Dec. 2017.
• [36] Moritz Hardt, Eric Price, and Nati Srebo, “Equality of Opportunity in Supervised Learning,” in Advances in Neural Information Processing Systems 29 (NIPS 2016), 2016.
• [37] M. B. Zafar, I. Valera, M. G. Rodriguez, and K. P. Gummadi, “Learning Fair Classifiers,” arXiv:1507.05259 [cs, stat] , Jul. 2015.
• [38] M. B. Zafar, I. Valera, M. G. Rodriguez, and K. P. Gummadi, “Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment,” arXiv:1610.08452 [cs, stat] , Oct. 2016.
• [39] R. S. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork, “Learning Fair Representations.,” ICML (3), vol. 28, pp. 325–333, 2013.
• [40] M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian, “Certifying and removing disparate impact,” in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015, pp. 259–268.
•