A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain,...
Transcript of A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain,...
![Page 1: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/1.jpg)
Intelligent Tutoring Systems A Logical Perspective
Sumit Gulwani
Microsoft Research, Redmond
July 2012
![Page 2: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/2.jpg)
• Application domain – Software Developers
– End-Users
– Education
• Specification of intent – Logic
– Examples
– Natural Language
• Search techniques – SAT/SMT solvers (Formal Methods)
– A*-style goal-directed search (AI)
– Version space algebras (Machine Learning)
1
Background: Program Synthesis
WAMBSE 2012: “Synthesis from Examples”, Gulwani.
![Page 3: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/3.jpg)
• Interactive Feedback – Answer-scripts graded immediately.
– Students provided feedback during solution generation
• Exploration – Interactive Visualization Tools
– Simulation Tools
• Social – Students working on same problem at same time can come
together.
– Learning experience for past students can be used to benefit the learning experience for future students.
2
Value Proposition
![Page 4: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/4.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
3
Intelligent Tutoring Systems
![Page 5: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/5.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
4
Intelligent Tutoring Systems
![Page 6: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/6.jpg)
Given a triangle XYZ, construct circle C such that C passes through X, Y, and Z.
5
Ruler/Compass based Geometry Constructions
X
Z
Y
L1 L2
N
C
PLDI 2011: Gulwani, Korthikanti, Tiwari.
![Page 7: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/7.jpg)
Given a triangle XYZ, construct circle C such that C passes through X, Y, and Z.
Precondition:
Slope(X,Y) Slope(X,Z) Æ Slope(X,Y) Slope(Z,X)
Postcondition:
LiesOn(X,C) Æ LiesOn(Y,C) Æ LiesOn(Z,C)
Where LiesOn(X,C) ´
Distance(X,Center(C)) = Radius(C)
Formal specification of the problem
6
![Page 8: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/8.jpg)
Geometry Program: A straight-line composition of geometry methods.
Geometry Types: Point, Line, Circle
Geometry Methods:
• Ruler(Point, Point) -> Line
• Compass(Point, Point) -> Circle
• Intersect(Circle, Circle) -> Pair of Points
• Intersect(Line, Circle) -> Pair of Points
• Intersect(Line, Line) -> Point
7
Solution is a program!
![Page 9: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/9.jpg)
Given a triangle XYZ, construct circle C such that C passes through X, Y, and Z.
8
Example of Geometry Program
1. C1 = Compass(X,Y); 2. C2 = Compass(Y,X); 3. <P1,P2> = Intersect(C1,C2); 4. L1 = Ruler(P1,P2); 5. D1 = Compass(Z,X); 6. D2 = Compass(X,Z); 7. <R1,R2> = Intersect(D1,D2); 8. L2 = Ruler(R1,R2); 9. N = Intersect(L1,L2); 10. C = Compass(N,X);
X
Z
Y
C1 C2 P1
P2
L1
D2
D1 R1
R2
L2
N
C
![Page 10: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/10.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
9
Intelligent Tutoring Systems
![Page 11: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/11.jpg)
10
Classic Problem in Automata Theory Course
Solution Generation Engine
Let L be the language containing all strings over {a,b} that have the same number of occurrences of “ab” as occurrences of “ba”. Construct an automata that accepts L, or prove that L is non-regular.
“Regular”, <Automata for L>
Joint work with Cerny, Henzinger, Zufferey, Radhakrishna
![Page 12: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/12.jpg)
11
Classic Problem in Automata Theory Course
Solution Generation Engine
Let L be the language containing all strings over {a,b} that have the same number of occurrences of “a” as occurrences of “b”. Construct an automata that accepts L, or prove that L is non-regular.
“Non-regular”, <Proof of non-regularity>
![Page 13: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/13.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
12
Intelligent Tutoring Systems
Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy
![Page 14: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/14.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
13
Intelligent Tutoring Systems
![Page 15: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/15.jpg)
14
Solution Generation by Example
Joint work with Erik Andersen and Zoran Popovic
![Page 16: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/16.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
15
Intelligent Tutoring Systems
![Page 17: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/17.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
16
Intelligent Tutoring Systems
![Page 18: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/18.jpg)
17
Background: PexForFun
![Page 19: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/19.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i < a.Length; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
18
Buggy Program for Array Reverse
6:28::50 AM
![Page 20: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/20.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length-1; i < a.Length-1; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
19
Buggy Program for Array Reverse
6:32::01 AM
![Page 21: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/21.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length-1; i < a.Length-1; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
20
Buggy Program for Array Reverse
6:32::32 AM
No change! Sign of Frustation?
![Page 22: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/22.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i <= a.Length; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
21
Buggy Program for Array Reverse
6:33::19 AM
![Page 23: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/23.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i < a.Length; i--)
{ Console.Writeline(i);
b[count] = a[i];
count++;
}
return b;
} }
22
Buggy Program for Array Reverse
6:33::55 AM
Same as initial attempt except Console.Writeline!
![Page 24: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/24.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i < a.Length; i--)
{ Console.Writeline(i);
b[count] = a[i];
count++;
}
return b;
} }
23
Buggy Program for Array Reverse
6:34::06 AM
No change! Sign of Frustation?
![Page 25: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/25.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i <= a.Length; i--)
{ Console.Writeline(i);
b[count] = a[i];
count++;
}
return b;
} }
24
Buggy Program for Array Reverse
6:34::56 AM
The student has tried this before!
![Page 26: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/26.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i < a.Length; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
25
Buggy Program for Array Reverse
6:36::24 AM
Same as initial attempt!
![Page 27: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/27.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length-1; i < a.Length-1; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
26
Buggy Program for Array Reverse
6:37::39 AM
The student has tried this before!
![Page 28: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/28.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i > 0; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
27
Buggy Program for Array Reverse
6:38::11 AM
Almost correct! (a[i-1] instead of a[i] in loop body)
![Page 29: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/29.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i >= 0; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
28
Buggy Program for Array Reverse
6:38::44 AM
Student going in wrong direction!
![Page 30: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/30.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i < a.Length; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
29
Buggy Program for Array Reverse
6:39::33 AM
Back to bigger error!
![Page 31: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/31.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i < a.Length; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
30
Buggy Program for Array Reverse
6:39::45 AM
No change! Frustation!
![Page 32: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/32.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i < a.Length; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
31
Buggy Program for Array Reverse
6:40::27 AM
No change! More Frustation!!
![Page 33: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/33.jpg)
using System;
public class Program {
public static int[] Puzzle(int[] a) {
int[] b = new int[a.Length];
int count = 0;
for(int i=a.Length; i < a.Length; i--)
{
b[count] = a[i];
count++;
}
return b;
} }
32
Buggy Program for Array Reverse
6:40::57 AM
No change! Too Frustated now!!! Gives up.
![Page 34: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/34.jpg)
Provides additional value over counterexample feedback.
• More friendly feedback. – Helpful for students who give up after several tries
(with only counterexample feedback).
• Grading – Counterexample feedback does not distinguish
between a slightly incorrect solution and one that is very far off from being correct.
33
Proposal: Semantic Grading
![Page 35: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/35.jpg)
Array Index: v[a] -> v[{a+1, a-1}]
Increment: v++ -> { ++v, v--, --v }
Conditional: a op b -> a ops b
where ops = { <, >, <=, >=, ==, != }
Initialization: v=n -> v={n+1, n-1, 0}
Return Value: return v -> return ?v
34
Example Errors
Joint work with Rishabh Singh and Armando Solar-Lezama
![Page 36: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/36.jpg)
Automated Grading (Array Reverse)
i = 1
i <= a.Length
--back
front <= back
![Page 37: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/37.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
36
Intelligent Tutoring Systems
![Page 38: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/38.jpg)
Example Problem: sec 𝑥 + cos 𝑥 sec 𝑥 − cos 𝑥 = tan2 𝑥 + sin2 𝑥
Query: 𝑇1 𝑥 ± 𝑇2(𝑥) 𝑇3 𝑥 ± 𝑇4 𝑥 = 𝑇5
2 𝑥 ± 𝑇62(𝑥)
𝑇1≠ 𝑇5
New problems generated:
csc 𝑥 + cos 𝑥 csc 𝑥 − cos 𝑥 = cot2 𝑥 + sin2 𝑥 (csc 𝑥 − sin 𝑥)(csc 𝑥 + sin 𝑥) = cot2 𝑥 + cos2 𝑥 (sec 𝑥 + sin 𝑥)(sec 𝑥 − sin 𝑥) = tan2 𝑥 + cos2 𝑥
:
37
Trigonometry Problem
AAAI 2012: Rohit Singh, Sumit Gulwani, Sriram Rajamani
![Page 39: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/39.jpg)
38
Algebra Problem Generation
Example Problem
Query Generation
Query
Query Execution
New Problems
Query Refinement Results OK?
Refined Query
No
Yes
Similar Problems
![Page 40: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/40.jpg)
Example Problem: lim𝑛→∞ 2𝑖2 + 𝑖 + 1
5𝑖= 5
2
𝑛
𝑖=0
Query: lim𝑛→∞ 𝐶0𝑖2 + 𝐶1𝑖 + 𝐶2
𝐶3𝑖
= 𝐶4𝐶5
𝑛
𝑖=0
C0≠ 0 ∧ gcd 𝐶0, 𝐶1, 𝐶2 = gcd 𝐶4, 𝐶5 = 1
New problems generated:
lim𝑛→∞ 3𝑖2 + 2𝑖 + 1
7𝑖= 7
3
𝑛
𝑖=0
lim𝑛→∞ 3𝑖2 + 3𝑖 + 1
4𝑖= 4
𝑛
𝑖=0
lim𝑛→∞ 𝑖2
3𝑖= 3
2
𝑛
𝑖=0
lim𝑛→∞ 5𝑖2 + 3𝑖 + 3
6𝑖= 6
𝑛
𝑖=0
39
Limits/Series Problem
![Page 41: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/41.jpg)
Example Problem: (csc 𝑥) (csc 𝑥 − cot 𝑥) 𝑑𝑥 = csc 𝑥 − cot 𝑥
Query: 𝑇0 𝑥 𝑇1 𝑥 ± 𝑇2 𝑥 𝑑𝑥 = 𝑇4 𝑥 ± 𝑇5(𝑥)
𝑇1≠ 𝑇2 ∧ 𝑇4 ≠ 𝑇5
New problems generated:
(tan 𝑥) (cos 𝑥 + sec 𝑥) 𝑑𝑥 = sec 𝑥 − cos 𝑥
(sec 𝑥) (tan 𝑥 + sec 𝑥) 𝑑𝑥 = sec 𝑥 + cot 𝑥
(cot 𝑥) (sin 𝑥 + csc 𝑥) 𝑑𝑥 = sin 𝑥 − csc 𝑥 40
Integration Problem
![Page 42: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/42.jpg)
Ex. Problem 𝑥 + 𝑦 2 𝑧𝑥 𝑧𝑦
𝑧𝑥 𝑦 + 𝑧 2 𝑥𝑦
𝑦𝑧 𝑥𝑦 𝑧 + 𝑥 2 = 2𝑥𝑦𝑧 𝑥 + 𝑦 + 𝑧 3
Query 𝐹0(𝑥, 𝑦, 𝑧) 𝐹1(𝑥, 𝑦, 𝑧) 𝐹2(𝑥, 𝑦, 𝑧)𝐹3(𝑥, 𝑦, 𝑧) 𝐹4(𝑥, 𝑦, 𝑧) 𝐹5(𝑥, 𝑦, 𝑧)𝐹6(𝑥, 𝑦, 𝑧) 𝐹7(𝑥, 𝑦, 𝑧) 𝐹8(𝑥, 𝑦, 𝑧)
= 𝐶10𝐹9(𝑥, 𝑦, 𝑧)
𝐹𝑖≔ 𝐹𝑗 𝑥 → 𝑦; 𝑦 → 𝑧; 𝑧 → 𝑥 𝑤ℎ𝑒𝑟𝑒 𝑖, 𝑗 ∈ { 4,0 , 8,4 , 5,1 ,… }
New problems generated:
𝑦2 𝑥2 𝑦 + 𝑥 2
𝑧 + 𝑦 2 𝑧2 𝑦2
𝑧2 𝑥 + 𝑧 2 𝑥2 = 2 𝑥𝑦 + 𝑦𝑧 + 𝑧𝑥 3
𝑦𝑧 + 𝑦2 𝑥𝑦 𝑥𝑦
𝑦𝑧 𝑧𝑥 + 𝑧2 𝑦𝑧
𝑧𝑥 𝑧𝑥 𝑥𝑦 + 𝑥2 = 4𝑥2𝑦2𝑧2
41
Determinant Problem
![Page 43: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/43.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
42
Intelligent Tutoring Systems
![Page 44: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/44.jpg)
1. The principal characterized his pupils as _________ because they were pampered and spoiled by their indulgent parents.
2. The commentator characterized the electorate as _________ because it was unpredictable and given to constantly shifting moods.
(a) cosseted
(b) disingenuous
(c) corrosive
(d) laconic
(e) mercurial
43
Sentence Completion Problems
Joint work with Geoffrey Zweig and Vikram Rao Sudarshan
![Page 45: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/45.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
44
Intelligent Tutoring Systems
![Page 46: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/46.jpg)
tan 3𝑥 tan 2𝑥 tan 𝑥 = tan 3𝑥 − tan 2𝑥 − tan 𝑥
𝑦𝑧 − 𝑥2 𝑧𝑥 − 𝑦2 𝑥𝑦 − 𝑧2
𝑧𝑥 − 𝑦2 𝑥𝑦 − 𝑧2 𝑦𝑧 − 𝑥2
𝑥𝑦 − 𝑧2 𝑦𝑧 − 𝑥2 𝑧𝑥 − 𝑦2
𝐴1 sin
3 𝛼 𝐵1 sin3 𝛽 𝐶1 sin
3 𝛾𝐴2 sin 𝛼 𝐵2 sin 𝛽 𝐶2 sin 𝛾
45
Mathematical (Syntactic) Intellisense
Joint work with Alex Polozov and Sriram Rajamani
![Page 47: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/47.jpg)
Prove (csc 𝑥 − sin 𝑥)(sec 𝑥 − cos 𝑥)(tan 𝑥 + cot 𝑥) = 1
L.H.S. = 1
sin 𝑥− sin 𝑥
1
cos 𝑥− cos 𝑥
sin 𝑥
cos 𝑥+cos 𝑥
sin 𝑥
= 1 − sin2 𝑥
sin 𝑥
1 − cos2 𝑥
cos 𝑥
sin2 𝑥 + cos2 𝑥
cos 𝑥 sin 𝑥
= cos2 𝑥
sin 𝑥
sin2 𝑥
cos 𝑥
1
cos 𝑥 sin 𝑥
= 1
46
Mathematical (Semantic) Intellisense
![Page 48: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/48.jpg)
• Various Aspects – Solution Generation
– Grading/Feedback
– Problem Generation
– Content Entry
• Various Domains – Geometry, Algebra, Physics, Chemistry, Logic
• Partnership with UW and IITK for deployment in schools
– Programming, Automata, Excel, SQL • Partnership with MIT, UPenn, UIUC for EdX/internal deployment
– Language Learning • Interest from Office Education Team
47
Intelligent Tutoring Systems
CHI 2012: Cheema, Gulwani, LaViola
![Page 49: A Logical Perspective - Computer Science · Intelligent Tutoring Systems Joint work with Jain, Singh, Sharang, Sharma, Karkare, Roy • Various Aspects – Solution Generation –](https://reader036.fdocuments.us/reader036/viewer/2022081405/5f0b5a8a7e708231d4301988/html5/thumbnails/49.jpg)
• Technical Perspective – Aspects: Solution+Problem Generation, Grading, Content Entry
– Domains: Math/Science, Programming, Language Learning
• Value Proposition: – Short term: Interactive Feedback, Exploration, Social
– Long Term: Ultra-intelligent computer, Model of human mind, Inter-stellar travel
• Crowdsourcing Opportunities – Solution Generation: Natural Language to Logic translation
– Problem Generation: Rate difficulty and ambiguity level
– Automated Grading: Provide natural explanations for errors
48
Conclusion