Finding Help with Programming Errors: An Exploratory Study
of Novice Software Engineers’ Focus in Stack Overflow Posts
Preetha Chatterjee Minji Kong Lori Pollock
Journal of Systems and Software (JSS), Volume 159, Jan 2020
Asst. Prof.
[Fall ’21]
Ph.D.
Stack Overflow Usage: Statistics and Challenges
40% participants are novice programmers (< 5 yrs experience)
who struggle to find relevant information from a Stack Overflow post!!
Information Overload:
The top 5 Stack Overflow
questions for a sample
query “Differences
Hashmap Hashtable Java”,
consist of 51 answers
which have 6,771 words in
total [Xu et. al. 2017]
https://insights.stackoverflow.com/survey/2020
We aim to understand:
How novice software engineers direct their efforts and what kinds of
information they focus on within a Stack Overflow post
Study Design
RQ1. Challenges - What slows a
novice software engineer down in
identifying the solution most
appropriate for their problem?
RQ2. Focus - Which parts of a post
novice software engineers focus on to
reduce their time in locating
information?
RQ1 Findings: Developer Challenges
Survey
Question
69% participants indicated “Too much text containing unnecessary details”
51% participants indicated “Determining embedded code segment relevance”
39% indicated “Too much code containing unnecessary details”
Question Title: Polymorphism in Java error: cannot find Symbol
I've just started learning object oriented programming from the book head first java.
It said that polymorphism enables me to create an array of the superclass type and
then have all the subclasses as the array elements.
But when I tried writing code using the same principles it ran into error saying error:
cannot find symbol I made the classes the superclass was animal and the dog class
extended the animal class having a fetch method of its own, but when I referenced
the dog variable as animal it did not work here is the code
…
The error
tester.java:4: error: cannot find symbol doggie.fetch();
symbol: method fetch()
location: variable doggie of type animal
When using polymorphism, if you create an instance of the subclass and store its
reference in a variable of superclass type, you can only call those methods on the
newly created instance which are present in the super class.
In your code, you created an instance of dog class and stored its reference
in doggie which is of type animal (super class of dog), In such case, you can't call any
method on dog class instance that isn't available in animal class. fetch method is not
defined in the animal class hence you get the error.
Either define the fetch method in the animal class
OR Change animal doggie = new dog(); to dog doggie = new dog();
Developer’s Goal
Developer’s Pertinent
Action
System Symptom
Accepted Answer
Symptom Cause
Posted Solution
Solution Justification
RQ2 Findings: Developer Focus
• Too much text including unnecessary
details (69%)
• Determining embedded code segment
relevance (51%)
• Too much code including unnecessary
details (39%)
Developer’s Goal
Developer’s Pertinent Action
System Symptom
Symptom Cause
Posted Solution
Solution Justification
Novice software engineers pay attention to only 27% code and 15-21% of text in a SO post!
@PreethaChatterj
preethac@udel.edu sites.udel.edu/preethac/
Significance:
 improve the Q&A forum interface
 guide tools for mining forums
 improve granularity of traceability mappings involving forum posts
Paper link: https://doi.org/10.1016/j.jss.2019.110454
Summary and Overview
RQ1. Challenges - What slows a
novice software engineer down in
identifying the solution most
appropriate for their problem?
RQ2. Focus - Which parts of a post
novice software engineers focus on to
reduce their time in locating
information?

Finding Help with Programming Errors: An Exploratory Study of Novice Software Engineers’ Focus in Stack Overflow Posts

  • 1.
    Finding Help withProgramming Errors: An Exploratory Study of Novice Software Engineers’ Focus in Stack Overflow Posts Preetha Chatterjee Minji Kong Lori Pollock Journal of Systems and Software (JSS), Volume 159, Jan 2020 Asst. Prof. [Fall ’21] Ph.D.
  • 2.
    Stack Overflow Usage:Statistics and Challenges 40% participants are novice programmers (< 5 yrs experience) who struggle to find relevant information from a Stack Overflow post!! Information Overload: The top 5 Stack Overflow questions for a sample query “Differences Hashmap Hashtable Java”, consist of 51 answers which have 6,771 words in total [Xu et. al. 2017] https://insights.stackoverflow.com/survey/2020 We aim to understand: How novice software engineers direct their efforts and what kinds of information they focus on within a Stack Overflow post
  • 3.
    Study Design RQ1. Challenges- What slows a novice software engineer down in identifying the solution most appropriate for their problem? RQ2. Focus - Which parts of a post novice software engineers focus on to reduce their time in locating information?
  • 4.
    RQ1 Findings: DeveloperChallenges Survey Question 69% participants indicated “Too much text containing unnecessary details” 51% participants indicated “Determining embedded code segment relevance” 39% indicated “Too much code containing unnecessary details”
  • 5.
    Question Title: Polymorphismin Java error: cannot find Symbol I've just started learning object oriented programming from the book head first java. It said that polymorphism enables me to create an array of the superclass type and then have all the subclasses as the array elements. But when I tried writing code using the same principles it ran into error saying error: cannot find symbol I made the classes the superclass was animal and the dog class extended the animal class having a fetch method of its own, but when I referenced the dog variable as animal it did not work here is the code … The error tester.java:4: error: cannot find symbol doggie.fetch(); symbol: method fetch() location: variable doggie of type animal When using polymorphism, if you create an instance of the subclass and store its reference in a variable of superclass type, you can only call those methods on the newly created instance which are present in the super class. In your code, you created an instance of dog class and stored its reference in doggie which is of type animal (super class of dog), In such case, you can't call any method on dog class instance that isn't available in animal class. fetch method is not defined in the animal class hence you get the error. Either define the fetch method in the animal class OR Change animal doggie = new dog(); to dog doggie = new dog(); Developer’s Goal Developer’s Pertinent Action System Symptom Accepted Answer Symptom Cause Posted Solution Solution Justification RQ2 Findings: Developer Focus
  • 6.
    • Too muchtext including unnecessary details (69%) • Determining embedded code segment relevance (51%) • Too much code including unnecessary details (39%) Developer’s Goal Developer’s Pertinent Action System Symptom Symptom Cause Posted Solution Solution Justification Novice software engineers pay attention to only 27% code and 15-21% of text in a SO post! @PreethaChatterj preethac@udel.edu sites.udel.edu/preethac/ Significance:  improve the Q&A forum interface  guide tools for mining forums  improve granularity of traceability mappings involving forum posts Paper link: https://doi.org/10.1016/j.jss.2019.110454 Summary and Overview RQ1. Challenges - What slows a novice software engineer down in identifying the solution most appropriate for their problem? RQ2. Focus - Which parts of a post novice software engineers focus on to reduce their time in locating information?

Editor's Notes

  • #3 The recent Stack Overflow survey shows that 40% participants are novice programmers i.e. people with less than 5 years of coding experience. SO often contain long threads of discussions. A previous study suggests that reading answers of a popular SO post could take about 30 minutes!! Because of this information overload, Finding help from SO is especially difficult for novice software engineers. We conducted an exploratory study to understand how novice software engineers direct their efforts and what kinds of information they focus on within a post.
  • #4 Specifically, our exploratory case study answers 2 research questions: To answer RQ1, we conducted a survey to gather novice software engineers’ perceptions of what slows them down in their Stack Overflow usage. For (RQ2) the software engineers first annotated 400 Stack Overflow posts to indicate what they would recommend to highlight to focus attention. Then, we analyzed their annotations, to gain more insight into their recommendations.
  • #5 Findings from RQ1 suggest that, majority of participants indicated the obstacles of “Too much text containing unnecessary details”, followed by “Determining embedded code segment relevance”, and “Too much code containing unnecessary details”.
  • #6 The results from RQ2 discern the kinds of information prominent in the developers’ focus. In the question we find the: Developers’ Goal: or their intended outcome. Developer’s Pertinent Action: that led to the error. Observed system response to the developer’s action. In the answer we find the: cause of the error. Suggested approach to fix the error. Reason why the suggested Solution works.
  • #7 The results from this study can be leveraged to guide tools for mining forums, and improve granularity of traceability mappings involving forum posts.