Post on 23-Dec-2015
Activities Review Process
On the Cutting Edge Workshop on Teaching Hydrogeology, Soils, and Low-T Geochemistry in the 21st Century
HSG Review Management Team
Devin CastendykSUNY, Oneonta
Managing Editor
John McDarisScience Education Resource Center (SERC)
Barbara TewksburyHamilton College
Cutting Edge PI
Maddy SchreiberVirginia Tech
Associate Editor
The collections SERC: Science Education Resource Center
at Carleton CollegeSERC office and staff helps develop and
manage web resources for many projects through collaboration
Many different collections of activities, submitted for different projects
On the Cutting Edge: the first of the projects hosted by SERCMost of the activities in the Cutting Edge
collections were submitted in connection with workshops.
Cutting Edge Reviewed Collection
On the Cutting Edge is conducting a review of activities in the Cutting Edge collections
Each activity reviewed twice and ranked:Exemplary (Part of Reviewed Collection)Pass (Part of Reviewed Collection)Keep (Not part of Reviewed Collection)De-accession
Cutting Edge Reviewed Collection
Activities ranked as “Exemplary”Come up first in searchesAre designated on individual the
ActivitySheet as being part of Exemplary Teaching Activities collection
Cutting Edge Reviewed CollectionActivities ranked as “Pass”
Come up second in searchesAre designated on individual ActivitySheet
as being part of the Peer Reviewed Teaching Activities collection
Cutting Edge Reviewed CollectionActivities ranked as “Keep”
Come up last in searchesHave no designation on the ActivitySheetWorth keeping as catalyst idea
Might be too local or items might be missing or has other problematic aspects
Review process
Any activity tagged with hydro, soils, low-T geochem, biogeochem, or ES that has not been reviewed yet
Each activity receives 2 reviews Authors of “Exemplary” and “Pass” activities
receive letters Explains review process Indicates activity rank Indicates that reviewer comments are available
on request if the author wishes to revise If author does revise, the activity will be reviewed
again
Plan for webinar
Explain the web interfaceClarify review criteriaAnswer questions
As we go along, please post questions in chat
Your list of itemsWhen you click on Review Tool on
Review Team Instructions page, your login will take you to a page that lists only your items to review
http://serc.carleton.edu/NAGTWorkshops/hydrogeo/HSG2013/review_team.html
Your list of itemsIf you have not yet completed the
review, you will see:
Click the URL to go to the ActivitySheet and download the actual activity and any supporting materials.
The review tool
You will evaluate the activity in five categoriesScientific accuracyAlignment of goals, activity, and assessmentPedagogic effectivenessRobustnessActivity description
The review toolSummary score will tabulate automatically
Exemplary = 4; Very good = 3; Adequate = 2; Problematic = 1
Comments help the editors understand your ranking – please don’t leave these boxes blank!!
The review tool At the end of the form, you will add your view
about what it would take to raise the activity to Exemplary status if it fell short in your review
The editors will use your comments to respond to authors on request. Please phrase your comments in a collegial fashion.
Your list of items
Once you have submitted a review, your review list indicates completion for that item
Can you revise a review? Yes – click on the Review It link and then
the link to what you submitted previously
Your original rankings will come up, and you can change them and add to/change your comments.
Click submit when done.
Your reviewReview the activity in the context for
which it was designedNot just whether it’s good for a particular
upper level course – many will be for other courses (e.g., intro geo)
Not everything has to be a full lab or major assignment (e.g., a back-of-the-envelope calculation could be Exemplary)
Not every activity needs to be usable by all instructors (e.g., a lab requiring specific software/math/expertise background)
Your review
Make a summary list of the activities that you reviewed
List both the total numerical score and your overall assessmentExemplary“Exemplary minus”PassKeepDe-accession
Bring with you to review team meeting
Your review
Exemplary Must have good science, good pedagogy,
and all materials so that someone else can adapt/adopt, nothing “broken”
Can be “local” if it is also a good templateDoes not need to have answer key or to
provide an instructor with backgroundWe have never required these so cannot ding
someone for not including them
ScoringExemplary or very good in all categoriesExemplary in at least three of the five.18 or higher.
Your reviewExemplary minus
Could be made Exemplary with only a small amount of work, such as:fixing a URLuploading the latest version of the
assignment or adding instructor tipsfleshing out the ActivitySheet
This is not a formal category, but it would help us a lot to have your list of “Exemplary Minus” activities
These will be ranked as Pass, but knowing that they are “Exemplary minus” will help the editors craft feedback if authors request it.
Your review
Pass – these become part of the Reviewed CollectionThose that aren’t Exemplary but still have
value to othersMust be more than just the germ of an idea Must have all of the components
These must have no scientific errors. If you think there are errors, confirm this with
someone else on the review team. Those with scientific errors should go into the
Keep or De-accession category, depending on
the severity of the problem.
Your review
Keep – no designation on ActivitySheet, will come up last in a searchNucleus of a good ideaInsufficient info for someone to adapt or
adopt or has scientific errorsAuthor does not receive a letter
Summary Review each activity using rubric Score the activity in each of 5 categories Write a summary evaluation for each
Remember that these were submitted voluntarily to a community collection
Be kind but clear Make a summary list
ExemplaryExemplary minus (technically a Pass)PassKeepDe-accession
Your assignment
Each team member has 10-11 activities to review
Reviews must be completed and submitted using review tool before the review team meeting in ABQ
Meeting in ABQ on June 4 at 8:30 amDiscuss issuesResolve discrepant reviewsArrive at final rankings