https://www.codehunt.com

Teaching and Learning
Programming and
Software Engineering
via Interactive Gaming

http://pex4fun.com

Tao Xie
University of Illinois at Urbana-Champaign
In collaboration with Nikolai Tillmann, Jonathan de Halleux, and Judith Bishop
(Microsoft Research)
Related Papers/Resources: https://sites.google.com/site/asergrp/projects/ese
Testing Tool  Educational Gaming

Support

Dynamic Symbolic Execution (Pex)
http://research.microsoft.com/pex/

Pex for Fun:
Interactive Gaming for
Teaching and Learning

https://www.codehunt.com
http://pex4fun.com
Agenda
Background
Coding Duels
Educational Platform

Experiences
Conclusion
Agenda
Background
Coding Duels
Educational Platform

Experiences
Conclusion
Background
Dynamic Symbolic Execution (DSE)
aka. Concolic Testing
[Godefroid et al. 05][Sen et al. 05][Tillmann et al. 08]

Instrument code to explore feasible paths

http://research.microsoft.com/pex/
http://research.microsoft.com/pex/
Dynamic Symbolic Execution in Pex
Choose next path
Solve

void CoverMe(int[] a)
{
if (a == null) return;
if (a.Length > 0)
if (a[0] == 1234567890)
throw new Exception("bug");
}

F

F

a.Length>0

a==null

Execute&Monitor

Constraints to solve

Input

Observed constraints

a!=null

null
{}

a==null
a!=null &&
!(a.Length>0)
a==null &&
a.Length>0 &&
a[0]!=1234567890
a==null &&
a.Length>0 &&
a[0]==1234567890

a!=null &&
a.Length>0
a!=null &&
a.Length>0 &&
a[0]==123456890

T

T

{0}

{123…}

Done: There is no path left.

a[0]==123…

F

T

http://pex4fun.com/HowDoesPexWork
Coding Duels

1,463,179

clicked 'Ask Pex!'
Coding Duels

Pex computes “semantic diff” in cloud
secret reference implementation vs.
code written in browser
You win when Pex finds no differences
For more info, see our ICSE 2013 SEE paper:
http://web.engr.illinois.edu/~taoxie/publications/icse13see-pex4fun.pdf
Behind the Scene of Pex for Fun
behavior

Secret Impl ==
Secret Implementation
class Secret {
public static int Puzzle(int x) {
if (x <= 0) return 1;
return x * Puzzle(x-1);
}
}

class Test {
public static void Driver(int x) {
if (Secret.Puzzle(x) != Player.Puzzle(x))
throw new Exception(“Mismatch”);
}
}

Player Impl

Player Implementation

class Player {
public static int Puzzle(int x) {
return x;
}
}

10
Coding Duels
Fun and Engaging
Iterative gameplay
Adaptive
Personalized
No cheating
Clear winning criterion
Social Experience
Community
High score lists, leaderboard
Live feed

http://pex4fun.com/Community.aspx
http://pex4fun.com/Livefeed.aspx
Agenda
Background
Coding Duels
Educational Platform

Experiences
Conclusion
Teaching and Learning
Skills Being Trained
Induction
Problem solving/debugging
Program understanding/programming
Testing
Specification writing
Coding Duels for Course Assignments
@Grad Software Engineering Course

Observed Benefits
• Automatic Grading
• Real-time Feedback (for Both Students and Teachers)
• Fun Learning Experiences
http://pexforfun.com/gradsofteng
Coding Duel Competition

@ICSE 2011

http://pexforfun.com/icse2011
Example User Feedback
Released since 2010

“I used to love the first person shooters and the
satisfaction of blowing away a whole team of
Noobies playing Rainbow Six, but this is far more
fun.”

X

“I’m afraid I’ll have to constrain myself to spend just an hour
or so a day on this really exciting stuff, as I’m really stuffed
with work.”
“It really got me *excited*. The part that got me most is
about spreading interest in teaching CS: I do think that it’s
REALLY great for teaching | learning!”
Usage Scenarios of Pex4Fun
• Massive Open Online Courses (MOOC)
• Course assignments (students/professionals)
• Student/professional competitions

• Assessment of testing, programming,
problem solving skills for job applicants
Code Hunt: Resigned As Game
https://www.codehunt.com/
End-User Programming
On Mobile Devices

TouchDevelop:
Bring your own devices,
program anywhere,
run immediately on devices

V.S.

https://www.touchdevelop.com/

21
TouchDevelop

https://www.touchdevelop.com/

22
TouchDevelop

https://www.touchdevelop.com/

23
TouchDevelop

https://www.touchdevelop.com/

24
TouchDevelop

https://www.touchdevelop.com/

25
TouchDevelop cont.
• Simple script language
– No user-defined type
– No class inheritance

Full-fledged platform for
• Rich API support
EUP on Mobile Devices

– Easy access to various sensors and resources

• Powerful cloud services
– Download/publish scripts
– Store information of entities (e.g., scripts, users)
– Answer queries
https://www.touchdevelop.com/

26
Rapid Increase of TouchDevelop Popularity
Oct 2013 vs. 17322 scripts Feb 2013

27
Summary: Testing Tool Educational Gaming

Support

DSE/Pex

Pex for Fun:

Interactive Gaming for
Teaching and Learning
https://www.touchdevelop.com/

https://www.codehunt.com
http://pex4fun.com
Testing Tool  Educational Gaming
http://research.microsoft.com/pex/

http://pex4fun.com

Support

Q&A
Thank you!
Contact: taoxie@illinois.edu
http://www.cs.illinois.edu/homes/taoxie/
Related Papers/Resources: https://sites.google.com/site/asergrp/projects/ese
Summary: Testing Tool Educational Gaming

Support

DSE/Pex

Pex for Fun:

Interactive Gaming for
Teaching and Learning
https://www.touchdevelop.com/

https://www.codehunt.com
http://pex4fun.com

Teaching and Learning Programming and Software Engineering via Interactive Gaming

  • 1.
    https://www.codehunt.com Teaching and Learning Programmingand Software Engineering via Interactive Gaming http://pex4fun.com Tao Xie University of Illinois at Urbana-Champaign In collaboration with Nikolai Tillmann, Jonathan de Halleux, and Judith Bishop (Microsoft Research) Related Papers/Resources: https://sites.google.com/site/asergrp/projects/ese
  • 2.
    Testing Tool Educational Gaming Support Dynamic Symbolic Execution (Pex) http://research.microsoft.com/pex/ Pex for Fun: Interactive Gaming for Teaching and Learning https://www.codehunt.com http://pex4fun.com
  • 3.
  • 4.
  • 5.
    Background Dynamic Symbolic Execution(DSE) aka. Concolic Testing [Godefroid et al. 05][Sen et al. 05][Tillmann et al. 08] Instrument code to explore feasible paths http://research.microsoft.com/pex/
  • 6.
  • 7.
    Dynamic Symbolic Executionin Pex Choose next path Solve void CoverMe(int[] a) { if (a == null) return; if (a.Length > 0) if (a[0] == 1234567890) throw new Exception("bug"); } F F a.Length>0 a==null Execute&Monitor Constraints to solve Input Observed constraints a!=null null {} a==null a!=null && !(a.Length>0) a==null && a.Length>0 && a[0]!=1234567890 a==null && a.Length>0 && a[0]==1234567890 a!=null && a.Length>0 a!=null && a.Length>0 && a[0]==123456890 T T {0} {123…} Done: There is no path left. a[0]==123… F T http://pex4fun.com/HowDoesPexWork
  • 8.
  • 9.
    Coding Duels Pex computes“semantic diff” in cloud secret reference implementation vs. code written in browser You win when Pex finds no differences For more info, see our ICSE 2013 SEE paper: http://web.engr.illinois.edu/~taoxie/publications/icse13see-pex4fun.pdf
  • 10.
    Behind the Sceneof Pex for Fun behavior Secret Impl == Secret Implementation class Secret { public static int Puzzle(int x) { if (x <= 0) return 1; return x * Puzzle(x-1); } } class Test { public static void Driver(int x) { if (Secret.Puzzle(x) != Player.Puzzle(x)) throw new Exception(“Mismatch”); } } Player Impl Player Implementation class Player { public static int Puzzle(int x) { return x; } } 10
  • 11.
    Coding Duels Fun andEngaging Iterative gameplay Adaptive Personalized No cheating Clear winning criterion
  • 12.
    Social Experience Community High scorelists, leaderboard Live feed http://pex4fun.com/Community.aspx http://pex4fun.com/Livefeed.aspx
  • 13.
  • 14.
  • 15.
    Skills Being Trained Induction Problemsolving/debugging Program understanding/programming Testing Specification writing
  • 16.
    Coding Duels forCourse Assignments @Grad Software Engineering Course Observed Benefits • Automatic Grading • Real-time Feedback (for Both Students and Teachers) • Fun Learning Experiences http://pexforfun.com/gradsofteng
  • 17.
    Coding Duel Competition @ICSE2011 http://pexforfun.com/icse2011
  • 18.
    Example User Feedback Releasedsince 2010 “I used to love the first person shooters and the satisfaction of blowing away a whole team of Noobies playing Rainbow Six, but this is far more fun.” X “I’m afraid I’ll have to constrain myself to spend just an hour or so a day on this really exciting stuff, as I’m really stuffed with work.” “It really got me *excited*. The part that got me most is about spreading interest in teaching CS: I do think that it’s REALLY great for teaching | learning!”
  • 19.
    Usage Scenarios ofPex4Fun • Massive Open Online Courses (MOOC) • Course assignments (students/professionals) • Student/professional competitions • Assessment of testing, programming, problem solving skills for job applicants
  • 20.
    Code Hunt: ResignedAs Game https://www.codehunt.com/
  • 21.
    End-User Programming On MobileDevices TouchDevelop: Bring your own devices, program anywhere, run immediately on devices V.S. https://www.touchdevelop.com/ 21
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
    TouchDevelop cont. • Simplescript language – No user-defined type – No class inheritance Full-fledged platform for • Rich API support EUP on Mobile Devices – Easy access to various sensors and resources • Powerful cloud services – Download/publish scripts – Store information of entities (e.g., scripts, users) – Answer queries https://www.touchdevelop.com/ 26
  • 27.
    Rapid Increase ofTouchDevelop Popularity Oct 2013 vs. 17322 scripts Feb 2013 27
  • 28.
    Summary: Testing ToolEducational Gaming Support DSE/Pex Pex for Fun: Interactive Gaming for Teaching and Learning https://www.touchdevelop.com/ https://www.codehunt.com http://pex4fun.com
  • 29.
    Testing Tool Educational Gaming http://research.microsoft.com/pex/ http://pex4fun.com Support Q&A Thank you! Contact: taoxie@illinois.edu http://www.cs.illinois.edu/homes/taoxie/ Related Papers/Resources: https://sites.google.com/site/asergrp/projects/ese
  • 30.
    Summary: Testing ToolEducational Gaming Support DSE/Pex Pex for Fun: Interactive Gaming for Teaching and Learning https://www.touchdevelop.com/ https://www.codehunt.com http://pex4fun.com