Gustavo Pinto, Fernando KameiFederal University of Pernambuco

{ghlp, fkk}@cin.ufpe.br

What Programmers Say About Refactoring Tools?

An Empirical Investigation of StackOverflow

ProgrammersResearchers

ProgrammersResearchers

What they think about?

ProgrammersResearchers

What they think about?

ProgrammersResearchers

What they think about?

1.5M Users3.4M Questions6.8M Answers40GB of data

A TON of information(with some quality)

1.5M Users3.4M Questions6.8M Answers40GB of data

But, what about Refactoring Tools?

A TON of information(with some quality)

Filter questions by “refactoring” and “tools” on the body, or

subject or tag name.

Store the results in a single MySQL table.

Manual filter to remove false positives

The final data!

3.4M Questions

754 Q>2K A

from July 31, 2008 to July 31, 2012:

● 324 Questions● 1,135 Answers● 1,254 Users

Filter questions by “refactoring” and “tools” key-word, on the

body, or subject or tag name.

Store the results in a single MySQL table.

Manual filter to remove false positives

The final data!

3.4M Questions

754 Q>2K A

from July 31, 2008 to July 31, 2012:

● 324 Questions● 1,135 Answers● 1,254 Users

Filter questions by “refactoring” and “tools” key-word, on the

body, or subject or tag name.

Store the results in a single MySQL table.

Manual filter to remove false positives

The final data!

3.4M Questions

754 Q>2K A

from July 31, 2008 to July 31, 2012:

● 324 Questions● 1,135 Answers● 1,254 Users

2x

Research Questions

● RQ1: What are the most desirable features in refactoring tools?

● RQ2: What are the barriers to adoption of refactoring tools?

● RQ3: How is interest in refactoring tools related to programmer expertise?

● RQ4: Does interest in refactoring tools increase over the years?

1. Refactoring for dynamic languages (78 questions ≈ 24.72%)a. PHP » 29%b. JavaScript » 23%c. Python » 23%d. Ruby » 11%e. Perl » 9%f. R » 1%

g. Perl » 1%h. Matlab » 1%

RQ1: What are the most desirable features in refactoring tools?

StackOverflow Question #9424189

“I am looking for a python tool that will allow me to specifically rename and move

modules.”

1. Refactoring for dynamic languages (78 questions ≈ 24.72%)2. Refactoring recommendations (45 questions ≈ 13.88%)

“I would prefer a tool that just makes suggestions about possible refactorings: names the refactoring, optionally provides a short description of it (great for learning purposes),

highlights the code section and lets me do the refactoring myself”

RQ1: What are the most desirable features in refactoring tools?

StackOverflow Question #785667

RQ1: What are the most desirable features in refactoring tools?

“I would like to rename a table column in MySql and also have the name updated in any triggers, stored procs, etc. that reference the column. [...] If a tool exists that can perform such a

refactoring, it would be great if it could generate an SQL script that applies these changes.”

1. Refactoring for dynamic languages (78 questions ≈ 24.72%)2. Refactoring recommendations (45 questions ≈ 13.88%)3. Refactoring for databases (12 questions ≈ 3.71%)

a. Automatic simplification of SQLb. Refactoring table or views structurec. Fixing SQL vulnerabilities

StackOverflow Question #1644271

RQ1: What are the most desirable features in refactoring tools?

“I have inherited significant amounts of Groovy code, and I have found it difficult to maintain [..]. I have tried to manually convert some pieces to Java, it's been a pain. Are you aware of any tools

or plugins that help with this conversion?”

StackOverflow Question #5302103

1. Refactoring for dynamic languages (78 questions ≈ 24.72%)2. Refactoring recommendations (45 questions ≈ 13.88%)3. Refactoring for databases (12 questions ≈ 3.71%)4. Multi-language Refactorings (9 questions ≈ 2.77%)

a. All questions related to JVM languages

RQ2: What are the barriers to adoption of refactoring tools?

In order of interest:

1. Usability problems (52 questions ≈ 16%) 1.1. Unknown tool (54%)1.2. Difficult to learn (13%)1.3. Difficult to use (8%)

2. Lack of trust (39 questions ≈ 12%)

RQ2: What are the barriers to adoption of refactoring tools?

33% are from novice users

In order of interest:

1. Usability problems (16% of Q) 1.1. Unknown tool (54%)1.2. Difficult to learn (13%)1.3. Difficult to use (8%)

What refactoring tool do you use to do X in

language Z?

64%

49%

RQ2: What are the barriers to adoption of refactoring tools?

33% are from novice users

In order of interest:

1. Usability problems (16% of Q) 1.1. Unknown tool (54%)1.2. Difficult to learn (13%)1.3. Difficult to use (8%)

What refactoring tool do you use to do X in

language Z?

64%

49%

Barriers or lack of experience?

RQ2: What are the barriers to adoption of refactoring tools?

1. Usability problems (16% of Q) 1.1. Unknown tool (54%)1.2. Difficult to learn (13%)1.3. Difficult to use (8%)

2. Lack of trust (12 % of Q)

In order of interest:

Sometimes the tool will change the meaning of your code, without you expecting it.

StackOverflow Question #1372059

RQ2: What are the barriers to adoption of refactoring tools?

1. Usability problems (16% of Q) 1.1. Unknown tool (54%)1.2. Difficult to learn (13%)1.3. Difficult to use (8%)

2. Lack of trust (12 % of Q)

In order of interest:

Sometimes the tool will change the meaning of your code, without you expecting it.

StackOverflow Question #1372059

“It is not good to decrease readability in favor of atomicity”

StackOverflow Question #5302103

RQ3: How is interest in refactoring tools related to programmer expertise?

Or.. do only hackers talk about refactoring tools?

RQ3: How is interest in refactoring tools related to programmer expertise?

Or.. do only hackers talk about refactoring tools?

Q: 39% comes from novicesA: 41% comes from seniors

Novices Seniors

RQ3: How is interest in refactoring tools related to programmer expertise?

Or.. do only hackers talk about refactoring tools?

Q: 39% comes from novicesA: 41% comes from seniors

Novices Seniors

Refactoring tool is a topic that interest all kind of users.

RQ3: How is interest in refactoring tools related to programmer expertise?

Or.. do only hackers talk about refactoring tools?

Q: 39% comes from novicesA: 41% comes from seniors

Novices Seniors

Also, we ran a Person Correlation between reputation x questions and reputation x answers, but no significant correlation was found.

RQ4: Does interest in refactoring tools increase over the years?

Questions per month● Median: 6● Mean: 6.75● SD: 3.38● Max: 20● Min: 1

Answers per month● Median: 18● Mean: 23.63● SD: 16.23● Max: 73● Min: 1

● Yes, but very little.

RQ4: Does interest in refactoring tools increase over the years?

Questions per month● Median: 6● Mean: 6.75● SD: 3.38● Max: 20● Min: 1

Answers per month● Median: 18● Mean: 23.63● SD: 16.23● Max: 73● Min: 1

RQ4: Does interest in refactoring tools increase over the years?

Questions per month● Median: 6● Mean: 6.75● SD: 3.38● Max: 20● Min: 1

Answers per month● Median: 18● Mean: 23.63● SD: 16.23● Max: 73● Min: 1

● Yes, but very little.● An increment of 0.56 questions per trimester.● A decrement of -8.13 answers per trimester.

RQ4: Does interest in refactoring tools increase over the years?

Questions per month● Median: 6● Mean: 6.75● SD: 3.38● Max: 20● Min: 1

Answers per month● Median: 18● Mean: 23.63● SD: 16.23● Max: 73● Min: 1

● Yes, but very little.● An increment of 0.56 questions per trimester.● A decrement of -8.13 answers per trimester.

-0.43 without outliers

Conclusions● We provided a set of desired features, as well as common

cause for barriers adoption.○ Insights for researchers and tool vendors

● We observed that novice users produce more questions, and seniors provide the answers.○ Also, most of the barriers were reported by novice users

● We identified a relative small interest in refactoring tools ○ Mean of 6 questions / month, sd of 4.45

Gustavo Pinto, Fernando KameiFederal University of Pernambuco

{ghlp, fkk}@cin.ufpe.br

What Programmers Say About Refactoring Tools?

An Empirical Investigation of StackOverflow