CragCrunch: Insight Data Science Project
-
Upload
amy-skerry -
Category
Technology
-
view
390 -
download
1
Transcript of CragCrunch: Insight Data Science Project
My Memory is a bit sketchy on some of the pitches....feel free to offer corrections.... !!P1 11b slabby, bolt protected face crux starting off of a platform. !P2 11b Thin, left arching crack undercling/layback. Lots of fixed gear !P3 10a Slightly flaring thin corner, !P4 Big Traverse. !P5 10a Cool 10a arete/bulge move following by incredible 5.8 climbing, on atypically Yosemite sculpted jugs which turn out to be the trademark of the climb. Pay attention here, easy to get on the wrong crack. !Link Pitches 6&7 with a 70m rope(68m)....but save some gear for the anchor! !P6+P7 11c Long thin stemming corner with a oddly placed anchor at the crux. This will be the route crux for most. This anchor is not the end, keep going past the crux to the belay ledge above. With a 70m, you will have little rope to spare. !P8 11c A fantastic short but strenous 30' (?) overhanging hand and finger crack;
My Memory is a bit sketchy on some of the pitches....feel free to offer corrections.... !!P1 11b slabby, bolt protected face crux starting off of a platform. !P2 11b Thin, left arching crack undercling/layback. Lots of fixed gear !P3 10a Slightly flaring thin corner, !P4 Big Traverse. !P5 10a Cool 10a arete/bulge move following by incredible 5.8 climbing, on atypically Yosemite sculpted jugs which turn out to be the trademark of the climb. Pay attention here, easy to get on the wrong crack. !Link Pitches 6&7 with a 70m rope(68m)....but save some gear for the anchor! !P6+P7 11c Long thin stemming corner with a oddly placed anchor at the crux. This will be the route crux for most. This anchor is not the end, keep going past the crux to the belay ledge above. With a 70m, you will have little rope to spare. !P8 11c A fantastic short but strenous 30' (?) overhanging hand and finger crack;
My Memory is a bit sketchy on some of the pitches....feel free to offer corrections.... !!P1 11b slabby, bolt protected face crux starting off of a platform. !P2 11b Thin, left arching crack undercling/layback. Lots of fixed gear !P3 10a Slightly flaring thin corner, !P4 Big Traverse. !P5 10a Cool 10a arete/bulge move following by incredible 5.8 climbing, on atypically Yosemite sculpted jugs which turn out to be the trademark of the climb. Pay attention here, easy to get on the wrong crack. !Link Pitches 6&7 with a 70m rope(68m)....but save some gear for the anchor! !P6+P7 11c Long thin stemming corner with a oddly placed anchor at the crux. This will be the route crux for most. This anchor is not the end, keep going past the crux to the belay ledge above. With a 70m, you will have little rope to spare. !P8 11c A fantastic short but strenous 30' (?) overhanging hand and finger crack;
1) climb attributes (e.g. # of pitches) & unstructured text
2) individual user ratings 3) overall popularity
Model features:
Data Acquisition + Storage
Exploration & Feature Processing
Predicting Individual User Ratings
User Recommendations
scraped >100,000 climbs, ~5000 unique users (w/ ratings,
grades, comments) from MountainProject.com
Climb Similarities
• Random Forest Classifier • Feature selection for final
algorithm based on tf-idf and RF feature importances
Product
1) climb attributes (e.g. # of pitches) & unstructured text
2) individual user ratings 3) overall popularity
Model features:
Data Acquisition + Storage
Exploration & Feature Processing
Predicting Individual User Ratings
User Recommendations
scraped >100,000 climbs, ~5000 unique users (w/ ratings,
grades, comments) from MountainProject.com
Climb Similarities
• Random Forest Classifier • Feature selection for final
algorithm based on tf-idf and RF feature importances
Product
1) climb attributes (e.g. # of pitches) & unstructured text
2) individual user ratings 3) overall popularity
Model features:
Data Acquisition + Storage
Exploration & Feature Processing
Predicting Individual User Ratings
User Recommendations
scraped >100,000 climbs, ~5000 unique users (w/ ratings,
grades, comments) from MountainProject.com
Climb Similarities
• Random Forest Classifier • Feature selection for final
algorithm based on tf-idf and RF feature importances
Product
1) climb attributes (e.g. # of pitches) & unstructured text
2) individual user ratings 3) overall popularity
Model features:
Data Acquisition + Storage
Exploration & Feature Processing
Predicting Individual User Ratings
User Recommendations
scraped >100,000 climbs, ~5000 unique users (w/ ratings,
grades, comments) from MountainProject.com
Climb Similarities
• Random Forest Classifier • Feature selection for final
algorithm based on tf-idf and RF feature importances
Product
Data Acquisition + Storage
Exploration & Feature Processing
Predicting Individual User Ratings
User Recommendations Climb Similarities
Product
Personalized Recommendations
• flexibly identify candidates based on user-specified
constraints
• RF predicted ratings used to rank climbs based on
user’s personal preference history
Data Acquisition + Storage
Exploration & Feature Processing
Predicting Individual User Ratings
User Recommendations Climb Similarities
Product
Validation
Data Acquisition + Storage
Exploration & Feature Processing
Predicting Individual User Ratings
User Recommendations Climb Similarities
Product
Validation
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
true
posi
tive
rate
false positive rate
Data Acquisition + Storage
Exploration & Feature Processing
Predicting Individual User Ratings
User Recommendations Climb Similarities
Product
final model classifies individual user four-star ratings of different climbs
with 70.4% accuracy
reliably above chance and outperforms model based on
average popularity alone (56%)
Validation
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
true
posi
tive
rate
false positive rate
What features or dimensions of climbs determine individual differences in preferences?
• to better understand the model and diagnose diverse preferences
What features or dimensions of climbs determine individual differences in preferences?
• to better understand the model and diagnose diverse preferences• to integrate new users (with no rating history) by mapping them into a
reduced space of features
What features or dimensions of climbs determine individual differences in preferences?
• to better understand the model and diagnose diverse preferences• to integrate new users (with no rating history) by mapping them into a
reduced space of features
Brain & Cognitive Sciences MIT
Harvard University Ph.D spring 2015
Yale University, 2010
CragCrun.chby Amy Skerry
Limitations
1. Didn't take into account variability in grading standards across region• The same grade can refer to very different difficulty levels across different areas
Limitations
1. Didn't take into account variability in grading standards across region• The same grade can refer to very different difficulty levels across different areas• Ideally, a personalized model can go beyond recorded/consensus grades when
suggesting skill-appropriate climbs
Limitations
1. Didn't take into account variability in grading standards across region• The same grade can refer to very different difficulty levels across different areas• Ideally, a personalized model can go beyond recorded/consensus grades when
suggesting skill-appropriate climbs• Unfortunately, it is difficult to estimate the region/area-level differences in grading
standards from the data, because some areas are more stiff vs. soft in the grading, but some areas also contain easier vs. harder climbs. Thus, a simple approach like normalizing by average difficulty of the region would not suffice
Limitations
1. Didn't take into account variability in grading standards across region• The same grade can refer to very different difficulty levels across different areas• Ideally, a personalized model can go beyond recorded/consensus grades when
suggesting skill-appropriate climbs• Unfortunately, it is difficult to estimate the region/area-level differences in grading
standards from the data, because some areas are more stiff vs. soft in the grading, but some areas also contain easier vs. harder climbs. Thus, a simple approach like normalizing by average difficulty of the region would not suffice
2. Didn't take into account relative difficulty assessment as a signal
Limitations
1. Didn't take into account variability in grading standards across region• The same grade can refer to very different difficulty levels across different areas• Ideally, a personalized model can go beyond recorded/consensus grades when
suggesting skill-appropriate climbs• Unfortunately, it is difficult to estimate the region/area-level differences in grading
standards from the data, because some areas are more stiff vs. soft in the grading, but some areas also contain easier vs. harder climbs. Thus, a simple approach like normalizing by average difficulty of the region would not suffice
2. Didn't take into account relative difficulty assessment as a signal• A major factor in enjoyment of a climb is fit to the climber's skill set (i.e. if there is
a climb rated 5.12 that is all crimps and I find it easy, I'll probably enjoy that climb more than a climb that is rated 5.10d but very overhung and difficult for me
Limitations
1. Didn't take into account variability in grading standards across region• The same grade can refer to very different difficulty levels across different areas• Ideally, a personalized model can go beyond recorded/consensus grades when
suggesting skill-appropriate climbs• Unfortunately, it is difficult to estimate the region/area-level differences in grading
standards from the data, because some areas are more stiff vs. soft in the grading, but some areas also contain easier vs. harder climbs. Thus, a simple approach like normalizing by average difficulty of the region would not suffice
2. Didn't take into account relative difficulty assessment as a signal• A major factor in enjoyment of a climb is fit to the climber's skill set (i.e. if there is
a climb rated 5.12 that is all crimps and I find it easy, I'll probably enjoy that climb more than a climb that is rated 5.10d but very overhung and difficult for me
• In principle, we should be able to use the perceived difficulty of a climb to an individual user as another signal relating to the climber's skills and preferences
Limitations
1. Didn't take into account variability in grading standards across region• The same grade can refer to very different difficulty levels across different areas• Ideally, a personalized model can go beyond recorded/consensus grades when
suggesting skill-appropriate climbs• Unfortunately, it is difficult to estimate the region/area-level differences in grading
standards from the data, because some areas are more stiff vs. soft in the grading, but some areas also contain easier vs. harder climbs. Thus, a simple approach like normalizing by average difficulty of the region would not suffice
2. Didn't take into account relative difficulty assessment as a signal• A major factor in enjoyment of a climb is fit to the climber's skill set (i.e. if there is
a climb rated 5.12 that is all crimps and I find it easy, I'll probably enjoy that climb more than a climb that is rated 5.10d but very overhung and difficult for me
• In principle, we should be able to use the perceived difficulty of a climb to an individual user as another signal relating to the climber's skills and preferences
3. Popularity is weighted equally
Limitations
1. Didn't take into account variability in grading standards across region• The same grade can refer to very different difficulty levels across different areas• Ideally, a personalized model can go beyond recorded/consensus grades when
suggesting skill-appropriate climbs• Unfortunately, it is difficult to estimate the region/area-level differences in grading
standards from the data, because some areas are more stiff vs. soft in the grading, but some areas also contain easier vs. harder climbs. Thus, a simple approach like normalizing by average difficulty of the region would not suffice
2. Didn't take into account relative difficulty assessment as a signal• A major factor in enjoyment of a climb is fit to the climber's skill set (i.e. if there is
a climb rated 5.12 that is all crimps and I find it easy, I'll probably enjoy that climb more than a climb that is rated 5.10d but very overhung and difficult for me
• In principle, we should be able to use the perceived difficulty of a climb to an individual user as another signal relating to the climber's skills and preferences
3. Popularity is weighted equally• Overall popularity should be weighted in a graded fashion depending on the
amount of personalized information available for a given user
What features determine route preferences?bolt, climb, right, belay, route, ledge, corner, anchor, climbing, foot, face, tree, gear,
rope, continue, rock, wall, facing, follow, start, small, past, base, trail, summit, section, large, cam, easy, rappel, rap, gully, dihedral, straight, arete, class, short, reach, finger,
head, dike, camalot, pro, step, obvious, lead, block, approach, fixed, wide, easier, west, clip, ramp, descent, lower, gain, south, double, rack, bolts, loose, north, buttress,
east, near, second, way, finish, angle, scramble, walk, protected, ground, harder, directly, climber, single, nice, set, bit, standard, pull, possible, feet, horizontal,
protection, low, climbed, clean, big, end, long, line, passing, couple, edge, cliff, sustained, higher, variation, final, hold, hard, area, difficult, descend, road, lieback, seam, anchors, protect, upper, huge, bring, pitches, sloping, cams, slightly, leaning,
awkward, pocket, high, runout, cool, formation, downclimb, use, using, direct, grade, vertical, slings, quality, classic, fun, mantle, main, shallow, good, piece, ll, guide,
place, super, middle, chain, trad, run, located, solid, excellent, fa, center, nuts, easily, pretty, probably, starts, great, leading, positive, point, follows, moves, slanting,
medium, far, shared, just, draws, somewhat, lots, crag, lip, need, maybe, rail, half, hands, look, fall, sure
flakes, hand cracks, tricky moves, underclings, slabs, clipping, technical climbs,
bouldery moves, bulge, steep climbs, juggy holds, chimney climbs, crack climbs, tough
cruxes, roofs, flakes, traverses, multi-pitch climbs,
crimps
PredictedTr
ue
flakes, hand cracks, tricky moves, underclings, slabs, clipping, technical climbs,
bouldery moves, bulge, steep climbs, juggy holds, chimney climbs, crack climbs, tough
cruxes, roofs, flakes, traverses, multi-pitch climbs,
crimps
Algorithm
• Analysis decisions/hyper-parameters determined based on exploratory analysis of New Hampshire climbs, final model and validation based on
California data
Algorithm
• Analysis decisions/hyper-parameters determined based on exploratory analysis of New Hampshire climbs, final model and validation based on
California data
Algorithm
• Analysis decisions/hyper-parameters determined based on exploratory analysis of New Hampshire climbs, final model and validation based on
California data
• For each individual user, trained a single model based on feature vectors derived from 3 sources: climb attributes (mostly a priori term frequencies from unstructured text), overall popularity, and individual preferences from
other climbers (binarized).
Algorithm
• Analysis decisions/hyper-parameters determined based on exploratory analysis of New Hampshire climbs, final model and validation based on
California data
• For each individual user, trained a single model based on feature vectors derived from 3 sources: climb attributes (mostly a priori term frequencies from unstructured text), overall popularity, and individual preferences from
other climbers (binarized).
Algorithm
• Analysis decisions/hyper-parameters determined based on exploratory analysis of New Hampshire climbs, final model and validation based on
California data
• For each individual user, trained a single model based on feature vectors derived from 3 sources: climb attributes (mostly a priori term frequencies from unstructured text), overall popularity, and individual preferences from
other climbers (binarized).
• Random Forest Classifer (k=80) used to classify individual star ratings. Recommendations generated by applying pre-trained model to new candidates based on constraints (e.g. area) specified by the user
Algorithm
• Analysis decisions/hyper-parameters determined based on exploratory analysis of New Hampshire climbs, final model and validation based on
California data
• For each individual user, trained a single model based on feature vectors derived from 3 sources: climb attributes (mostly a priori term frequencies from unstructured text), overall popularity, and individual preferences from
other climbers (binarized).
• Random Forest Classifer (k=80) used to classify individual star ratings. Recommendations generated by applying pre-trained model to new candidates based on constraints (e.g. area) specified by the user
Algorithm
• Analysis decisions/hyper-parameters determined based on exploratory analysis of New Hampshire climbs, final model and validation based on
California data
• For each individual user, trained a single model based on feature vectors derived from 3 sources: climb attributes (mostly a priori term frequencies from unstructured text), overall popularity, and individual preferences from
other climbers (binarized).
• Random Forest Classifer (k=80) used to classify individual star ratings. Recommendations generated by applying pre-trained model to new candidates based on constraints (e.g. area) specified by the user
• Features constructed and validated from personalization models also used to construct affinity matrix of all climbs (cosine similarity on text and user-based
feature vectors)
Validation
• Leave-one-out cross validation procedure (on separate model for each
user)
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7Accuracy
Validation
• Leave-one-out cross validation procedure (on separate model for each
user)
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7Accuracy
Validation
• Leave-one-out cross validation procedure (on separate model for each
user)
• Model comparison: for each model, compute separate accuracy score for each user assess & statistical
significance of model differences by testing
whether gain of model is reliable across users
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7Accuracy
Validation
• Leave-one-out cross validation procedure (on separate model for each
user)
• Model comparison: for each model, compute separate accuracy score for each user assess & statistical
significance of model differences by testing
whether gain of model is reliable across users
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7Accuracy
Validation
• Leave-one-out cross validation procedure (on separate model for each
user)
• Model comparison: for each model, compute separate accuracy score for each user assess & statistical
significance of model differences by testing
whether gain of model is reliable across users
• Model performance increases as a
function of the data available for the individual user
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7Accuracy