Electronic Supplementary Material for:

Analytical reasoning task reveals limits of social learning in networks

Iyad Rahwan, Dmytro Krasnoshtan, Azim Shari↵, Jean-Francois Bonnefon

A Experimental Interface

A.1 Tutorial Screenshots

1

2

3

4

5

A.2 Quiz Questions

Following is the set of quiz questions that participants have to correctly answer before they can proceed tothe experiment. The correct answers are highlighted in bold.

How many times will you see each question?

1. Only once

2. 5 times

Does your reward depends on the responses of other players?

1. Yes

2. No

How do you make money in this game?

1. Every correct response gets me money

2. Not every correct response gets me money, only the last trial counts for each question

3. Not every correct response gets me money, multiple correct responses to the same question only countas one

6

B Questions

Below is the list of all questions. The first three questions corresponded to the Cognitive Reflection Test(CRT). These three questions generate an incorrect intuitive answer, which must be inhibited in order toproduce the correct analytic answer [1].

1. [CRT1] In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 daysfor the patch to cover the entire lake, how long would it take for the patch to cover half of the lake?(Answer: 47)

2. [CRT2] If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines tomake 100 widgets? Write the answer in minutes.(Answer: 5)

3. [CRT3] A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much doesthe ball cost?(Answer: 0.05)

After the three CRT questions, subjects moved on to another series of four questions from the BerlinNumeracy Test (BNT), which we do not discuss in this article [2]. Being either too easy or too hard, thesequestions produced little variance between participants in our networks, and thus did not allow us to testour hypotheses (see Figure 1 below for a visualization of the responses in the BNT questions). As thesequestion came after participants had completed the three CRT questions, there is no concern that they couldhave contaminated the CRT data that we analyze in this article.

1. [BNT1] Imagine we are throwing a five-sided die 50 times. On average, out of these 50 throws howmany times would this five-sided die show an odd number (1, 3 or 5)? out of 50 throws.(Answer: 30)

2. [BNT2] Out of 1,000 people in a small town 500 are members of a choir. Out of these 500 membersin the choir 100 are men. Out of the 500 inhabitants that are not in the choir 300 are men. What isthe probability that a randomly drawn man (not a person) is a member of the choir? (please indicatethe probability in percents)(Answer: 25)

3. [BNT3] Imagine we are throwing a loaded die (6 sides). The probability that the die shows a 6 istwice as high as the probability of each of the other numbers. On average, out of these 70 throws, howmany times would the die show the number 6? out of 70 throws.(Answer: 20)

4. [BNT4] In a forest 20% of mushrooms are red, 50% brown and 30% white. A red mushroom ispoisonous with a probability of 20%. A mushroom that is not red is poisonous with a probability of5%. What is the probability that a poisonous mushroom in the forest is red?(Answer: 50)

7

First Question Second Question Third Question Fourth Qestion

0.00

0.25

0.50

0.75

1.00

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5Trial

Prop

ortio

n of

cor

rect

resp

onse

s

TOPOLOGY Full Erdos−Renyi Barabasi−Albert Clustered Baseline

Figure 1: BNT questions are either too easy or too hard, reducing variance observed in CRT questions.

8

C Network Structures & Their Properties

C.1 The Networks

A network (or graph) consists of a set V vertices (a.k.a. nodes, individuals), and a set E of edges (a.k.a.connections or links) between them. Elements of E can be denoted by a pair Eij = (i, j) where i, j 2 V .

Each of our experimental sessions ran on one of the four graphs: (1) Barabasi-Albert model; (2) Clusteringgraph; (3) Erdos-Renyi model; (4) Full graph. The di↵erent graph structures are visually depicted below.These graphs were chosen due to significant discrepancy in their measures on the macro (network) and micro(node) level, as shown below.

Barabasi−Albert model Clustering graph

Full graph Erdos−Renyi model

Figure 2: List of graphs

9

C.2 Micro Measures

And on the micro level (for each node):Degree: The degree ki of a vertex i is simply the number of edges incident to this vertex. In a directed

graph, we can distinguish between the out-degree (outgoing edges) and in-degree (incoming edges). In thepresent paper, edges are considered undirected.

The di↵erent graph structures we used have significantly varying distributions of node degrees, as shownbelow. The degree distribution of the Erdo-Renyi graph follows a Poission distribution, thus most nodeshave a more or less equal number of neighbors (no one is disproportionately popular). In contrasted, in theBarabasi-Albert graph, the distribution is long-tailed, with a few very highly connected nodes. In the fullgraph, everyone has equal degree, since everyone is connected to everyone else. Finally, in the clusteringgraph, degrees are more or less identical. However, as we will see below, some nodes are a more privilagedposition in the middle of the graph.

0 5 10 15 200

5

10

15

20Barabasi model

0 5 10 15 200

5

10

15

20Erdos−Renyi model

0 5 10 15 200

5

10

15

20Full graph

0 5 10 15 200

5

10

15

20Clustering graph

Figure 3: Degree distribution

Local clustering coe�cient: The local clustering coe�cient captures the following intuition: out of allpairs of friends that i is connected to, how many of those friends are also friends with one another. In otherwords:

Ci =number of triangles connected to node i

number of triples centered around node i

where a triple centred around node i is a set of two edges connected to node i (if the degree of node i is 0or 1, we which gives us Ci = 0/0, we can set Ci = 0). High local clustering coe�cient for node i indicatesthat i belongs to a tightly knit group.

More formally, the local clustering coe�cient ci is defined as follows:

Ci =|{Ejk}|

ki(ki � 1): vj , vk 2 Ni, Ejk 2 E

where ki is the out-degree of vertex i, and Ni = {vj : Eij 2 E} is the set of out-neighbours of vertex i. For

10

undirected graphs the value of Ci is normalized as C 0i = 2Ci. If to rephrase this in the simple words, the

local clustering coe�cient of a vertex in a graph shows how close its neighbors are to being a full graph.The figure below highlights how the distribution of local clustering coe�cients varies significantly across

the di↵erent network structures. In particular, nodes in the Erdos-Renyi and Barabasi-Albert graphs havemuch lower clustering compared to the Clustering graph. Note that in the full graph, every node has a localclustering coe�cient of 1, since everyone is connected to everyone else.

Betweenness centrality: The betweenness centrality of a node is equal to the number of shortestpaths (among all other vertices) that pass through that node. The higher the number, the more importantis the node, in the sense that there is a small number of hops between that node and the majority of thenetwork. Mathematically it can be defined as

g(v) =X

s 6=v 6=t

�st(v)

�st

where �st is the total number of shortest paths from node s to node t and �st(v) is the number of thosepaths that pass through v.

The figure below shows that the betweenness centrality of nodes in the Clustering graph vary significantly(contrast this with the fact that the node degrees in this graph are almost identical to one another).

0

0.5

1

Barabasi model Erdos−Renyi model Full graph Clustering graph

Clustering coefficient

−20

0

20

40

60

Barabasi model Erdos−Renyi model Full graph Clustering graph

Betweenness centrality

Figure 4: Clustering coe�cient, betweenness centrality

C.3 Macro Measures

Graph density: In graph theory, graph density is defined as 2|E||V |(|V |�1) . Density represents the ratio of

the number of edges to the maximum number of possible edges. Density will therefore have a value in theinterval [0, 1].Clustering coe�cient of a graph: The clustering coe�cient of an undirected graph is a measure of thenumber of triangles in a graph. The clustering coe�cient for the whole graph is the average of the localclustering coe�cients Ci:

C =1

n

nX

i=1

Ci

11

where n is the number of nodes in the network. By definition 0 Ci 1 and 0 C 1.Diameter: Diameter of the graph is the lenght of the longest shortest path between any two vertices of thegraph.

Macro level parameters for the four classes of networks are summarized in the table below. Note howthe density and diameter of all graphs is almost identical, with the exception of the full graph, which hasmaximum density.

graph type Density Clustering Diameter Number of edges

Barabassi 0.195 0.208 4 37Erdos-Renyi 0.211 0.158 4 40Full graph 1 1 1 190Clustering graph 0.179 0.714 5 34

12

D Evolution of Network States

The figures below show samples of the detailed evolution of correct (blue) and incorrect answers (red) in aselection of network/question combinations.

Figure 5: Evolution of the game (Barabasi-Albert, question 1)

13

Figure 6: Evolution of the game (Barabasi-Albert, question 3)

14

Figure 7: Evolution of the game (Full, question 1)

15

Figure 8: Evolution of the game (Full, question 2)

16

Figure 9: Evolution of the game (Erdos-Renyi, question 1)

17

References

[1] Frederick S. Cognitive reflection and decision making. The Journal of Economic Perspectives.2005;19(4):25–42.

[2] Cokely ET, Galesic M, Schulz E, Ghazal S, Garcia-Retamero R. Measuring risk literacy: The Berlinnumeracy test. Judgment and Decision Making. 2012;7(1):25–47.

18