jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated...

31
Hypothesis Prototypes: Overview (updated Sept 7, 2016) Overview A proposed approach Some questions for the team to discuss What kinds of things are, or are not, amenable to this approach? How do we decide which experiments to try? How do we engage with users to deliver and evaluate experiments? How can outcomes inform product development? How do we gather information about users’ experiences with prototypes? About this document _________________________________________ Summary of Experiments and Outcomes Slack notifications User Stories GItHub Repo Current Slack<->H integrations EDHD 7712 Beta Test Jon Udell <[email protected]> Remi Holden Jon Udell <[email protected]> Edt631 Beta Test Lessons learned Next steps (as of Aug 2016) Note to Tim Chan about the prototype: Group admin SCREENCAST: http://jonudell.net/h/group-admin-v1.mp4 Context User story The experiment

Transcript of jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated...

Page 1: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

Hypothesis Prototypes: Overview(updated Sept 7, 2016)

OverviewA proposed approachSome questions for the team to discuss

What kinds of things are, or are not, amenable to this approach?How do we decide which experiments to try?How do we engage with users to deliver and evaluate experiments?How can outcomes inform product development?How do we gather information about users’ experiences with prototypes?

About this document_________________________________________Summary of Experiments and OutcomesSlack notifications

User StoriesGItHub RepoCurrent Slack<->H integrationsEDHD 7712 Beta Test

Jon Udell <[email protected]>Remi HoldenJon Udell <[email protected]>

Edt631 Beta TestLessons learnedNext steps (as of Aug 2016)Note to Tim Chan about the prototype:

Group adminSCREENCAST: http://jonudell.net/h/group-admin-v1.mp4ContextUser storyThe experimentQuestions to answer

Does it solve an immediate problem and improve the workflow?How important is auditing and undoability?Should groups that can be managed in this way map to H groups or not?

OutcomesControlled tagging

Per-group tag namespacesUser storySCREENCAST: http://jonudell.net/h/controlled_tagging_v1.mp4OutcomesTech Notes

Sourcing controlled tags from an external ontology

Page 2: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

SCREENCAST: http://jonudell.net/h/controlled-tagging-with-scigraph-ontology.mp4Supplementing ngTagsInput with a custom app

SCREENCAST: http://jonudell.net/h/annotation-applet_03.mp4Faceted browsing/searchingExport

BLOGPOST: https://hypothes.is/blog/viewing-and-exporting-hypothesis-annotations/SCREENCAST: http://jonudell.net/h/export_just_selections.mp4

Copy annotation to clipboardSCREENCAST: http://jonudell.net/h/copy_paste_annotation_v1.mp4FEEDBACK: Terry Elliot

Highlight groups with annotationsSCREENCAST: http://jonudell.net/h/highlight-groups-with-annotations-v1.mp4

SciBot / EPMC IntegrationSCREENCAST: http://jonudell.net/h/scibot-epmc-v1.mp4

CanvasDOC: Installing the AppDOC: Using the AppDOC: Canvas Integration NotesDOC: Canvas Architecture NotesSCREENCAST: http://jonudell.net/h/canvas_oauth_v1.mp4SCREENCAST: http://jonudell.net/h/canvas_resource_selection_02.mp4SCREENCAST: http://jonudell.net/h/canvas_homework_submission.mp4

AAAS / DiscoverBLOGPOST: https://hypothes.is/blog/science-in-the-classroom/BLOGPOST: https://hypothes.is/blog/do-it-yourself-anchoring-and-the-evolution-of-the-hypothesis-toolkit/

SciBotBLOGPOST: https://blog.jonudell.net/2016/04/24/annotation-is-not-only-web-comments/

David KennedyBLOGPOST: https://hypothes.is/blog/annotating-to-extract-findings-from-scientific-papers/

Hacking the Hypothesis ClientDOC: A Ninja guide to hacking the Hypothesis ClientSCREENCAST: http://jonudell.net/h/annotation-applet_03.mp4

OverviewThe Manifesto for Agile Software Development lists 12 principles. Here’s #1:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

The experiments described here aim to align Hypothesis with that approach by helping us:

Page 3: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

- Collaborate with our users to discover and validate their stories- Use working prototypes as a key method of discovery and validation- Iterate rapidly on prototypes, delivering value at each step- Work on multiple user stories (in parallel) to clarify the all-up product roadmap

A proposed approachProposed: That we routinely articulate and validate user stories in collaboration with users. Such collaboration entails:

- Building simple prototypes that enable users to try a new thing -- crucially, in the context of their own live Hypothesis data/documents/groups/relationships.

- Inviting feedback- Iterating rapidly on prototypes as users react to them- Using what we learn from such preliminary design and development to inform final

design and development

Some questions for the team to discuss

What kinds of things are, or are not, amenable to this approach?Some things are purely internal. But if a thing is visible to a user -- or to the special kind of user who is a developer -- then it’s a candidate for “early and continuous delivery of valuable software.” Value means: “Helps the user do things that matter to the user.” The notifications and export prototypes are good examples. Neither is a polished feature but both are delivering value. Users of these prototypes are not using them in order to give us feedback, they are using them for benefits not yet available in the core product. A major corollary benefit is that we learn, from their interaction with these prototypes, much that will inform the features as they move into the core product.

How do we decide which experiments to try?Criteria include:

- How is it prioritized on the roadmap? Anything that ranks high on the engineering roadmap’s todo list (things that engineering will definitely be building) is by definition a candidate. But experiments out on the long tail matter too, especially if they’re fairly easy to do. The series of controlled tagging experiments is a good example of that. So: prioritize experiments in alignment with, but not in lockstep with, the product roadmap.

- How badly do users need it? Export, in particular, is a response to a strong, often-articulated need. Likewise the alt-stream and its current incarnation as faceted browser/viewer. So, prioritize what users need most.

- How much can product design and development learn? The design/dev team rightly wants its efforts to be guided by user stories. Prototypes can help us articulate those stories -- and evolve them -- in order to provide the best possible input to product design

Maryann Martone, 08/24/16,
We may even learn whether they should move into the core product or whether they are a specialized niche that is best supported in some other way.
Maryann Martone, 08/24/16,
Having some feature is often the difference between someone choosing to use Hypothes.is for a project or not.
Page 4: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

and development. So, prioritize where we can reap the most valuable inputs. Caveat: We don’t (and a priori can’t) know what experiments will prove most fruitful!

- What is the potential for synergy and reuse? We weren’t sure we could deliver homework submission as a feature of the Canvas prototype. But it turns out that a lightly-modified version of the export prototype meets the need for (a first iteration of) an artifact the student can submit and the teacher can evaluate in Speed Grader. It’s probably too simplistic to say: “Prioritize experiments that serve multiple purposes.” We didn’t anticipate reusing the export prototype in Canvas. But we should design every feature with an eye toward component integration, and prototypes can play a role there. Consider Activity Pages, for example. If we think about the feature as (among other things) a reusable component, then we can ask: “In what contexts, other than the main one we envision, can Activity Pages deliver value?” One answer should be: “In embedded widgets.” And the faceted-browser/export prototype has given us experience on that front as well. Our Letters to the Next President page, for example, embeds the export prototype as a widget.

How do we engage with users to deliver and evaluate experiments?One well-trodden path is our support process. Instead of saying: “It’s on the roadmap, we’ll get to it” we can instead say: “It’s on the roadmap, and we have a prototype that explores one approach [or several variants], would you like to try it and tell us what you think? The interaction with Tim Chan happened that way. Another good example was the interaction with Tressie Macmillan Cottom. Our engagement with her begins with the fact that we are following her, in #notifications on Slack, using the prototype Slack notifier, so we noticed when she did some awesome annotation of the Democratic speeches, offered her the export prototype, and got some nice PR on Twitter.

Maryann Martone, 08/24/16,
A much more customer-centered approach.
Maryann Martone, 08/24/16,
I think, as I think you are implying here, that what is developed for one use can often be re-purposed for another use. But we can't know that ahead of time.
Page 5: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

What’s more, that interaction led to the observation that the export subset we custom-made for her needed to be a thing she could do for herself, which yielded this iteration: http://jonudell.net/h/export_just_selections.mp4

Another well-trodden path: Partner relationships. With AAAS, David Kennedy, Andrew Su, NIF, AAAS, and Canvas users, we’ve responded to specific requests -- for group admin (AAAS), controlled tagging (Kennedy), even-more-custom controlled tagging (Su), standalone anchoring, group admin (AAAS), Canvas integration.

Delivery mechanisms so far:- Server-based app that uses the H API (e.g. the original alt-stream prototype, the Slack

notifier)- Client-based app that uses the H API (e.g. the current faceted viewer/browser, the

exporter)- Custom Chrome extensions (e.g. the group admin that AAAS is using)- Other flavors (e.g. the LTI app for Canvas)- Feature cohorts

How can outcomes inform product development?The primary benefit is validated user stories. We build the core product based on stories we adduce from our own intuition, from users’ reactions to the existing product, and from users’ requests to extend the product. “As X I want Y so that I can do Z” is a statement that can sometimes, maybe often, be tested early, with one or more cheap prototypes, so we can bring more cogent and refined user stories to final design and development.

Another key benefit is responsiveness. When we enable a thing (like export, or notifications) that won’t make it into the core product anytime soon, we win hearts and minds. For example:

The core design/dev process can’t -- and shouldn’t -- respond quickly to enhancement requests. But in some cases a complementary upstream process can. It takes pressure off the dev, users love us for it, and we learn invaluable things when we give them useful prototypes.

Page 6: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

How do we gather information about users’ experiences with prototypes?Cohorts have so far been small and so these experiments yield anecdotal responses, not quantitative data. Another way to say “anecdotal responses,” of course, is “user stories.” So: We capture feedback, we discuss it with users and amongst ourselves. It’s always a bonus if Hypothesis is the enabler for that information gathering, as nicely exemplified in the annotation activity surrounding http://jonudell.net/h/orphans-feedback/.

What are the goals of this information gathering? As above:

- To validate the user stories that drive design and development,- By meeting articulated needs with fast/simple/preliminary implementations

About this document

Above the fold, this document proposes an approach to satisfying customers with “early and continuous delivery of valuable software.”. Below the fold it summarizes a series of experiments that have used that approach to explore key roadmap deliverables in collaboration with our users. For each experiment, it describes (or will, when fully fleshed out):

- What need we addressed- What we hoped to learn by addressing it in a prototype- Who we are working with- What ideas / premises we are testing- How we are test those ideas- How users responded to the test with feedback- How the experiment evolved in response to feedback- What outcomes are useful input to product development

_________________________________________

Dan Whaley, 08/23/16,
One thing that is important is how we collect data. I know that eng and design in particular want as much direct feedback (comments, quotes, long form responses) as synthesis.A proposed section might cover how feedback is collected, what the goals are of it.
Jon Udell, 08/23/16,
Absolutely. That said, early feedback from /any/ users beats early feedback from no users!
Lena Gunn, 08/23/16,
This is key for me. How do we select users for testing? Expert users may give us one answer, new users (or people still not using H) may give us a complete different perspective.
Page 7: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

Summary of Experiments and OutcomesAbove the fold: rationale for (and discussion of) a proposed approach. Below the fold: summary of experiments and outcomes.

Page 8: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

Slack notificationsHypothesis staff have been seeing notifications in Slack for quite some time.

User StoriesAs a Hypothesis user, I want notifications to Slack when;

- An URL of interest to my team is annotated- A user of interest to my team annotates- A group of interest to my team is annotated

So that team members can be aware of that activity, and engage with it as needed.

GItHub RepoStefan Candea (coordinator, co-founder at European Investigative Collaborations) wants to use the kit for his project, so I’ve published to https://github.com/judell/h_notify

Current Slack<->H integrations

# notify when an url in a list of urls of interest to the team is annotated

urls = ['http://example.com', 'https://hypothes.is/blog/test-our-new-canvas-app-prototype/']for url in urls: notify_slack_url_activity(url=url, token=default_token, pickle='urls', channel='test', hook=default_hook)

# notify when there's an annotation from one among a list of users

users = ['remiholden', 'jeremydean']for user in users: notify_slack_user_activity(user=user, token=default_token, pickle='users', channel='test', hook=default_hook)

# notify when there's an annotation in a group

Page 9: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

notify_slack_group_activity(group='8gk9i7VV', groupname='Anyone Can Join', token=default_token, pickle='AnyoneCanJoin', channel='test', hook=default_hook)

# notify when there's an annotation with a specified tag

notify_slack_tag_activity(tag='nextprez', token=default_token, pickle='nextprez', channel='test', hook=default_hook)

EDHD 7712 Beta Test

Remi Holden’s class, 4 groups, each routing to its own Slack channel.

This is great Jon, thanks so much.

I set a few test annotations that came through for each of the four groups. And post-cache the most recent annotations from this afternoon/evening are starting to appear - it's nice.

The Slack channels are also tracking H replies, yes? I'm seeing, for example, a recent H reply in the Hufflepuff channel, but not two other H replies that I did about the same time (about 40 min ago) - those should have also come through, yes? Any idea why?

Jon Udell <[email protected]>

5:55 PM (16 hours

ago)

to Remi

> those should have also come through, yes? Any idea why?

Page 10: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

Not sure, if you relay the URLs of those annotations (or just the IDs, e.g. the F-UGLEhzEeaIKUc82tk_rA from hyp.is/F-UGLEhzEeaIKUc82tk_rA/web.mit.edu/evhippel/www/books/DI/DemocInn.pdf) I'll take a look.

One of the things I'm trying to work out with beta users is how to contextualize replies. With Jeff we wound up having another channel in his Slack to discuss that and other things, the upshot of our beta test is here: https://docs.google.com/document/d/1ubFV0UyAEs4xMijtASQl3zhVtG8nAE-tURVdlxXz1d8/edit#heading=h.eur9re2hy60m.

And here's where I'll aim to capture what we get out of this test: https://docs.google.com/document/d/1ubFV0UyAEs4xMijtASQl3zhVtG8nAE-tURVdlxXz1d8/edit#

Cheers,

Jon

Attachments area

Remi Holden

5:30 AM (5 hours ago)

to me

Wow Jon, this is quite the resource - thanks for sharing.

As for mapping replies, looking just at #hufflepuff_annotation:

- some replies did come through last night (4iIheHQ8EeaLk7OKg83PBg and KxH6nHQ-EeaZ3Cte1CVN8g)

- yet some i made yesterday afternoon did not (2qb2-HOXEeaitC96yfK-yQ and pz00qnOaEeaLfEs21o_qbw)

- and a second-level reply to a reply by a student also did not come through (pz00qnOaEeaLfEs21o_qbw)

- i just added another reply while writing this email (-o8JmHT0Eeaoz4fjU7ngvA), we'll see what happens

Page 11: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

And looking over your prior work with Jeff and Pagan, the following stood out:

"The annotation appears first, then replies, in order. If annotations from more than one doc were flowing into this channel, then it might not cohere as well. So one approach would be to try to capture more context on a per-notification basis. But another approach is to organize things so that there's one doc per channel. If that's convenient it might be the ideal approach."

In this case, we already have group-specific annotations from two docs flowing into the four channels. In fact, we'll have - at the least - 30 different docs feeding annotations as that corresponds with the readings in our course (https://arsdoctrina.wordpress.com/readings/). So as for easily capturing context in this arrangement... I need to think a bit more about that, as you all discussed, it's a tricky challenge.

Jon Udell <[email protected]>

10:43 AM (1 minute ago)

to Remi

All of the ids are saved on my end, so should have been sent, but who knows, lots of variables in play including me stopping/restarting the service a few times yesterday.

If you want to invite me into your Slack as an observer it'll help me understand how this plays out from your POV.

"So as for easily capturing context in this arrangement... I need to think a bit more about that, as you all discussed, it's a tricky challenge."

And quite possibly not one that can/should be primarily resolved in Slack. The attached image, for example, shows two views of student annotations for Alison Frazier's Machiavelli course at U Texas. These are by way of our Canvas app which uses a variant of the faceted browser/viewer I showed you. For each student, Alison sees only the threads that involve that student, and with the student's contributions highlighted within the thread. For example:

Page 12: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

That's a form of contextualization that wouldn't be doable, or even sensible, in Slack. So the trick will be -- as in all notification systems -- to strike the right balance. Deliver change notification often enough to help people stay connected, with enough context to help them decide when and how to drill down into the source system for details.

Edt631 Beta TestWe set up a notifier for Jeff Kupperman ([email protected]) and some of his colleagues at Univ of Mich Flint. Over a couple of days in June 2016 we iterated on the prototype based on their feedback.

pagan [1:44 PM] jkupp [1:19 PM]

@judell: That’s a great question about contextualizing notifications. Right now it’s a little hard to parse out

what the annotation was to. Would it be possible to include a little more of the context of the reference —

say, the highlighted word plus 30 or 40 characters before and after? Somehow visually highlighting the

original context and the annotation itself, and de-emphasizing the other info like the group name and

URL, which you want to be able to access when you need it, but probably not all the time... (edited)

pagan [1:22 PM]

Yes, but what about the students that add their annotation to the entire paragraph? That would clutter the

notification with a lot of text. I agree though about the group name and article link not being necessary to

each notification (another bonus of having article annotations go into a different channel).

jkupp [1:27 PM]

@pagan: Good point. So maybe a bit from before the annotation, and then if the reference (Or is it the

referent? The think the annotation is annotating) is more than a certain length, clip it and end the context

there; otherwise show a bit after the referent, too…. Something like that? In other words, give enough to

remind people reading the notification in Slack what part of the text is being annotated, without going back

Page 13: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

to the original. This presumes that everyone will have read the original at least once, but that’s probably a

fair assumption.

[1:29]

Or for longer referents, clip out the middle and display the beginning and end. E.g., “Four score and

seven years ago … shall not perish from the earth."

[1:30]

I love talking about these sort of things when I’m not the one who has to write the code, btw. Doesn’t

happen that much. :slightly_smiling_face:

judell [1:35 PM]

"I love talking about these sort of things when I’m not the one who has to write the code" And I love

prototyping things for a group of engaged and thoughtful users!

One thing I'm noodling on: include a textshot (http://leancrew.com/all-this/2016/01/twitter-textshots-and-

linking/) of the thread. Advantages could be: 1) a good way to deliver at-a-glance-context, 2) a universal

way to do it, would work in email or any other notification context as well. (edited)

pagan [1:40 PM]

That would be fantastic! (edited)

jkupp [1:41 PM]

@judell: Yes, something like that could be good … the challenge would be addressing Pagan’s point

about when people highlight a large chunk of text… and when a lot of people annotate the same bit or a

lot respond to the same annotation, you probably wouldn’t want dozens of copies of the same textshot in

the thread. Definitely an interesting design problem.

pagan [1:41 PM]

Hmmm

[1:42]

pagan [9:45 PM] @judell: fyi from one of my students - "This annotation slack integration is really cool. If you can pass along the following suggestion to the Hypothes.isfolks: Italicizing or color changing the quoted text and/or the response so that they do not flow together."

-----June 15th

-----

judell [7:45 AM] Thanks! I'll see what's possible with Slack formatting. And as I mentioned I also want to try a textshot approach that will work independently of Slack's formatting and the same way in any other notification environment too (e.g. email).

judell [9:34 AM] OK I've made some tweaks as shown here:

Page 14: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

judell [9:34 AM] uploaded an image: notification with reply

Add Comment

judell [9:46 AM] Thanks for the prompt to differentiate the elements. With respect to an individual notification, the changes are:

- introduce the notification as an `annotation` or a `reply`

- use doc title, not url, as text of the link to the annotated doc

- use blockquote formatting for the quote

- show tags in bold

The next question was how to contextualize replies. I had been thinking of ways to show context within a reply notification. But then I noticed that the replies were coming to Slack out of order. If I sort the results of the API query ascending by date, then we get a nice improvement. An `annotation`notification precedes any of its `reply` notifications. In the example above that's exactly what you want. Now, that's only true when the reply is the next thing in the result set. If there are intervening notifications the relationship won't be so clear. But it _may_ be workable to scan back in the notification stream and pick out the referent by matching the doc title. And that _might_ on the whole be more elegant than duplicating context within individual notifications that is also visible in the notification stream. So, let's give this a try and see how it feels.

jeremydean [10:02 AM] joined #hypothes_is_thoughts by invitation from @pagan

Page 15: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

-----June 16th

-----

jeremydean [7:55 AM] thanks so much for all your feedback here @pagan and @jkupp

[7:55]

chatted with remi holden last night--didn't realize you all were family!

jkupp [7:57 AM] Yes, it’s a little interconnected world we work in …. :slightly_smiling_face:

jkupp [8:04 AM] I like the blockquote formatting. The reply context is tricky … not obvious how to connect the reply to the annotation and also keep most recent notifications at the bottom of the stream. One thought, not sure if this is possible or even desirable, but in the reply could there be a link where the original annotation pops up on hover or is expandable on click? Then of course we probably want to see the context of other replies on the same annotation, too, if they exist… Hm….

pagan [8:15 AM] It's much better than it was. If the intent is to follow what's happening then this is working for me. Unless you get to the point where you can respond to annotations using Slack/email etc. then I think simple is better. You still want people to go back to the original page and annotate/reply.

[8:17]

@jeremydean: had a great visit with Remi last week when he was in BC. Got to see a picture of you two together in Berlin too. Definitely an interconnected world.

jkupp [8:18 AM] @pagan: That’s true — the main point is to let people know there’s something new on hypothes.is, so you probably don’t need to give people the whole context, just enough to understand what the new post is responding to.

Page 16: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

pagan [8:58 AM] It would be useful if there was a way to see what annotation thread the reply was from. Eg. Student annotates. Instructor comments later. Not clear in Slack channel which annotations that reply is addressing.

judell [12:10 PM] uploaded an image: Thread context

Add Comment

Page 17: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

judell [12:19 PM] The above example works out really nicely. The annotation appears first, then replies, in order. If annotations from more than one doc were flowing into this channel, then it might not cohere as well. So one approach would be to try to capture more context on a per-notification basis. But another approach is to organize things so that there's one doc per channel. If that's convenient it might be the ideal approach.

Lessons learned

Next steps (as of Aug 2016)As a user of the protoype I’m able to watch/follow users, sets of users, groups, urls, and sets of urls. It’s super-powerful. WA key challenge: Make it easy for a typical user to do that.

Note to Tim Chan about the prototype:Some things the prototype explores:

- Mapping Hypothesis queries to Slack channels. There are lots of way to do it. You can, for

example, route tags or groups or individual URLs or usernames to one or more Slack channels. We

also have channels that watch sets of URLs, and others that watch sets of usernames.

- Mapping Hypothesis annotations to Slack messages. We've iterated a few times, still aren't quite

sure how best to contextualize messages that belong to Hypothesis conversation threads.

Group admin

SCREENCAST: http://jonudell.net/h/group-admin-v1.mp4

We have a prototype extension that enables a group of “subject users” who have given us their H API keys to be subject to admin by an “admin user” who is the H user running the prototype extension. The admin user running the extension can edit or delete annotations by subject users.

Page 18: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

ContextIn phase 1 of Science in the Classroom we were able to give Melissa McCartney, the editor-in-chief, final say over annotations from a team of grad students. But it was super-awkward. The grad students annotated using their own accounts, then we used API code to transfer those annotations to Melissa’s account for final editing and approval. The Learning Lens code only fetches annotations from Melissa’s account (SitC).

User storyAs the editor in chief, I want to be able to edit annotations made by grad students to ensure they’re up to snuff before we release them. So that we’ll maintain quality control..

The experimentFor the next round they had already planned to skip that nonsense by sharing credentials for the SitC account, but that’s sub-optimal in several ways. So they’re going to give the prototype a try,

it should enable this workflow:- Students annotate- Melissa edits their annotations- Learning Lens fetches annotations directly from the student accounts

Questions to answer

Does it solve an immediate problem and improve the workflow?As per Outcomes (below), apparently yes.

How important is auditing and undoability?

Should groups that can be managed in this way map to H groups or not?In a related experiment, http://jonudell.net/h/controlled_tagging_v1.mp4, the notion is that controlled tags are made available to a group by that group’s owner/creator.

In this experiment, because AAAS do everything in Public, we can’t rely on H group membership. But that may be a useful constraint. It invites us to consider, for example, whether the scope of an H group is too restricted in some case, and whether/how to spread the subject/admin relationship across groups.

OutcomesSo far so good, Melissa is able to do what she needs to do. No more detailed feedback yet, I’ll

follow up when they’ve gone through their next production cycle.

Jon Udell, 07/28/16,
Additional questions welcome!
Page 19: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

Controlled taggingUser stories for this feature are many and diverse. We’ve tried a few variations so far.

Per-group tag namespaces

User storyDavid Kennedy is collaborating with others on a project called ReproNIM. When he switches to the Hypothesis group chosen for that collaboration, he wants a prescribed set of tags to populate the picklist. Examples: ReproNim:Resource, ReproNim:Training, ReproNim:RelatedLiterature.

SCREENCAST: http://jonudell.net/h/controlled_tagging_v1.mp4In this version, controlled namespaces are associated with groups. Switching to a group with an associated tag namespace binds that namespace to the tag editor. The namespaces are stored in a JSON file:

{"wqvn7Lar": ["Reading:Apples", "Reading:Oranges", "Reading:Grapes", "Reading:Pears", "Reading:Kumquats"],"GLPBM67j": ["Support:Peaches", "Support:Kumquats", "Support:Lemons"],"6iW9YXXP": ["Sail:Mast", "Sail:Bow", "Sail:Stern", "Sail:Keel"]}

OutcomesDavid hasn’t tried this yet, mainly because an earlier version of this idea is already meeting the need.

Tech Notes

The JSON file lives in Drive Notepad -- e.g. https://drivenotepad.github.io/app/?state={%22action%22:%22open%22,%22ids%22:[%220Bz4y9G08pmrYdkRXLV9aNnZQaHc%22]} -- which turns out to be a handy way to manage the config. Unlike Google Docs, it can serve a plain text file over HTTPS. Like Google Docs, it enable users (JSON-savvy users, admittedly) to edit the file themselves.

Maryann Martone, 08/24/16,
But I did.
Page 20: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

Sourcing controlled tags from an external ontology

SCREENCAST: http://jonudell.net/h/controlled-tagging-with-scigraph-ontology.mp4In this version, we bind the tag editor to an external ontology. It works out beautifully in this case because the SciGraph ontologies are served in a completion-friendly way.

Supplementing ngTagsInput with a custom app

SCREENCAST: http://jonudell.net/h/annotation-applet_03.mp4As discussed in the Nina Guide , intercepting ngTagsInput and connecting it to an external ontology -- though compelling, and undoubtedly useful in certain circumstances -- did not satisfy the Su Labs’ requirement to be able to contextualize the names in the picklist. V3 meets that requirement by enabling simple web apps to run in the context of annotation bodies -- an approach that also has much wider implications for democratizing H client development.

Faceted browsing/searching

Export

BLOGPOST: https://hypothes.is/blog/viewing-and-exporting-hypothesis-annotations/

SCREENCAST: http://jonudell.net/h/export_just_selections.mp4Note: A new requirement has emerged from our NIF partner. Anita Bandrowski wants to selectively delete annotations from the 3300 created by one of her curators. My response offers two approaches, one based on this prototype and another on the group admin prototoype.

Hey Jon, I may have missed this, but where is this in the list of features for Hypothes.is?Is there a hack yet on your server that I can use in the meantime??

---------- Forwarded message ----------From: Gabrielle M Pine <[email protected]>Date: Fri, Aug 19, 2016 at 11:37 AMSubject: Deleting AnotationsTo: [email protected]

Hey Anita,

I found some of Jessica's old annotations that need minor changes and I

Maryann Martone, 08/24/16,
Ninja?
Maryann Martone, 08/24/16,
SciGraph is a graph database for managing ontologies developed in part by NIF.
Page 21: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

was planning on remaking the annotations in the correct format anddeleting her incorrect ones. Can I get a feature that will allow me toselectively delete her annotations.

Best,Gabi

Hi Anita,

One way we could do it: give you a version of the group admin prototype, a custom extension that holds API keys for one or more users. (http://jonudell.net/h/group-admin-v1.mp4) The problem I foresee there: It's the H client, so you have to visit each annotated document to use the delete power on behalf of Jessica.

Another way: extend the export prototype (http://jonudell.net/h/export_just_selections.mp4) to delete, rather than export, selections. That'd still leave us with a page of 3300 checkboxes, but if there's enough info in the view to decide what to check, it might be the best route. Only problem there: no delete button. But it should be straightforward to add and would be a valuable thing for us to have anyway.

Copy annotation to clipboard

SCREENCAST: http://jonudell.net/h/copy_paste_annotation_v1.mp4

FEEDBACK: Terry ElliotI liked that there was a clipboard function. I thought there was too much cruft in what was copied. No need for initial time and other data. Just wanted the annotation. Also, I was hoping for a way to collect all of the annotations on a page in one clip on the clipboard. If you were able clip them all and multiple annotators were represented there it would look a little like Storify.

I would love a "copy all" button.

I sound truly picky, don't I? You asked so... I really love your /h page with all the coolities there. You are doing great things. BTW, have been checking out NowComment's new image annotation. They are so different than you guys, but I like you both.

Last, I have a dream one day that I will be able to click on a button on one of my annotations or one from my group and clone it into my wiki or wikity. That would be boss squared.

Page 22: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

Highlight groups with annotations

SCREENCAST: http://jonudell.net/h/highlight-groups-with-annotations-v1.mp4

SciBot / EPMC Integration

SCREENCAST: http://jonudell.net/h/scibot-epmc-v1.mp4

Canvas

DOC: Installing the App

DOC: Using the App

DOC: Canvas Integration Notes

DOC: Canvas Architecture Notes

SCREENCAST: http://jonudell.net/h/canvas_oauth_v1.mp4

SCREENCAST: http://jonudell.net/h/canvas_resource_selection_02.mp4

SCREENCAST: http://jonudell.net/h/canvas_homework_submission.mp4

AAAS / Discover

Page 23: jonudell.netjonudell.net/h/h_prototypes.docx  · Web viewHypothesis Prototypes: Overview (updated Sept 7, 2016) Overview. A proposed approach. Some questions for the team to discuss.

BLOGPOST: https://hypothes.is/blog/science-in-the-classroom/

BLOGPOST: https://hypothes.is/blog/do-it-yourself-anchoring-and-the-evolution-of-the-hypothesis-toolkit/

SciBot

BLOGPOST: https://blog.jonudell.net/2016/04/24/annotation-is-not-only-web-comments/

David Kennedy

BLOGPOST: https://hypothes.is/blog/annotating-to-extract-findings-from-scientific-papers/

Hacking the Hypothesis Client

DOC: A Ninja guide to hacking the Hypothesis Client

SCREENCAST: http://jonudell.net/h/annotation-applet_03.mp4