1 User Research for Improving Patent Examination Tools Marti Hearst Chief IT Strategist, USPTO (on...

1

User Research for Improving User Research for Improving Patent Examination ToolsPatent Examination Tools

Marti HearstMarti HearstChief IT Strategist, USPTOChief IT Strategist, USPTO

(on leave from UC Berkeley)(on leave from UC Berkeley)

October 26, 2010October 26, 2010

2

OutlineOutline

Recent Events at the USPTORecent Events at the USPTO The Patents End-to-End ProjectThe Patents End-to-End Project Results of User Research (so far)Results of User Research (so far)

– User interfaces generally– Search in particular

How can Researchers Help?How can Researchers Help?

3

Today, Change is the Today, Change is the Status Quo at the USPTOStatus Quo at the USPTO

…

4


5


In the last year:In the last year:– Changes in how examination is done– More hires of examiners with IP background– New ideas for accelerating certain patents’

reviews» Three-track examination, green tech pilot, humanitarian tech

– Increased transparency» Regular blog posts, Facebook» Patents Dashboard» Peer to Patent 2» (to appear) Discussion tools

– First Chief Economist for the PTO– Started Patents End-to-End IT!

6

The Current IT SystemThe Current IT System

Based on TIFF images rather than textBased on TIFF images rather than text Antiquated code base (Cobol)Antiquated code base (Cobol) Expensive to maintainExpensive to maintain Difficult to extendDifficult to extend Examiners use ~20 stove-piped interfacesExaminers use ~20 stove-piped interfaces

11

East/West Sample QueriesEast/West Sample Queries[COMMAND:Search] S(MRI.TI. AND ("324"/$).CCLS. AND "2010".PY.).#USOC,#USPT,#PGPB. [COMMAND:Search] S((historic near8 (information or regist$ or acqui$ or transmi$))).#pgpb,#uspt,. [COMMAND:Search] S("7220634".PN. AND 12).#USPT. [COMMAND:Search] S(historic).#pgpb,#uspt,#usoc,#epab,#jpab,#dwpi,#tdbd. [COMMAND:Search] S((information or regist$ or acqui$ or transmi$)).#pgpb,#uspt,#usoc,#epab,#jpab,#dwpi,#tdbd. [COMMAND:Search] S(3 AND (IDENTIF$4 OR ID)).#USOC,#DWPI,#TDBD,#EPAB,#JPAB,#FPRS,#USPT,#PGPB. [COMMAND:Search] S(((1) and (2))).#pgpb,#uspt,#usoc,#epab,#jpab,#dwpi,#tdbd. [COMMAND:Search] S(historic).#pgpb,#uspt,#usoc,#epab,#jpab,#dwpi,#tdbd. COMMAND:Search] S((((MONEY OR WIRE OR FUND) NEAR3 TRANSFER$4) WITH (SENDER AND (RECIPIENT OR RECEIVER))) AND ((RECIPIENT OR RECEIVER) WITH BIOMETRIC) AND @AD<"20070117").#USOC,#TDBD... [COMMAND:Search] S((information or regist$ or acqui$ or transmi$)).#pgpb,#uspt,#usoc,#epab,#jpab,#dwpi,#tdbd. [COMMAND:Search] S("6175752".PN.).#USPT,#PGPB. [COMMAND:Search] S(((PROGRAM$4 OR WRIT$4) WITH (SELECT$3 NEAR3 CELL) WITH VOLTAGE) SAME ((PROGRAM$4 OR WRIT$4) WITH (ADJACENT$3 NEAR5 CELL$1) WITH VOLTAGE) SAME (READ$3 WITH (SELECT$3 NEAR3 CELL)) SAME ((CHANG$4 OR OFFSET)NEAR4 VOLTAGE)).#DWPI,#TDBD... [COMMAND:Search] S(((PROGRAM$4 OR WRIT$4) WITH (SELECT$3 NEAR3 CELL) WITH VOLTAGE) SAME ((PROGRAM$4 OR WRIT$4) WITH (ADJACENT$3 NEAR5 CELL$1) WITH VOLTAGE) SAME (READ$3 WITH (SELECT$3 NEAR3 CELL)) SAME (OFFSET$4 NEAR4 VOLTAGE)).#DWPI,#TDBD,#EPAB,#JPAB,#USPT,#PGPB.

12

East/West Query Operator East/West Query Operator StatisticsStatistics

Sample sizeSample size 198031198031

ANDAND 129684129684

PNPN 6900769007

Wildcard ($)Wildcard ($) 3171131711

OROR 2818128181

NEARNEAR 1735317353

WITHWITH 1404014040

Class/subclassClass/subclass 1379613796

SAMESAME 1141611416

ADJADJ 1009710097

DATEDATE 46894689

NOTNOT 20432043

CLAIMCLAIM 17961796

XORXOR 33

13

The Vision for PE2EThe Vision for PE2E

Capabilities:Capabilities:Accept, manage, search, and publish XML-based

applicationsNew text search tools, and up-to-date version of EASTNew image search toolsNew pre-examination tools and linguistic analysis toolsIntegrated collaboration toolBased on open standardsFlexible, extensible, maintainable

Agile Development:Agile Development:Enhancements delivered in weeks rather than

months or years!

14

Designing in ParallelDesigning in Parallel

New SoftwareArchitecture

User Researchdesign IT right

Test Latest ToolsSearch, ling, viz, etc

ProcessRe-engineering

Prototype for “happy path,”then build version 1.0

Text and Image Search tool testing

Clickable prototypes for corefunctionality

Reviewing and streamlining all patent-related processes

15

design IT rightdesign IT right

User-Centered Design for the new User Interfaces

Business Goals:Survey (online or paper)

Tasks Analysis:Interviews, FocusGroups, and Online Input

Iterative TestingOf Mocked-up and Clickable Interface Prototypes

16

User Research To DateUser Research To Date

Focus groups and interviews with more Focus groups and interviews with more than 50 examinersthan 50 examiners

Focus groups with supervisorsFocus groups with supervisors Surveys on goals, search Surveys on goals, search (> 300 participants each)(> 300 participants each)

A timed study of old systemA timed study of old system Lots of free-text comments on blog posts Lots of free-text comments on blog posts

and surveysand surveys

17

What Examiners Want:One Seamless, Integrated UI

A system that is smart about their workA system that is smart about their workAutomatically fills in structured information

» e.g., bibliographic information

Automatically checks for consistency, errors, etc

» Warn if too many references used for this statute

Suggests queries, thesaurus terms, art to look at

» based on what this user has done in the past e.g.: automatically surface commonly used MPEP passages

» based on what others with similar goals have done

» based on an application or set of documents

18

What Examiners Want:One Seamless, Integrated UI

Allows annotation and sharingAllows annotation and sharingAbility to tag and annotate art, Office Actions, figures, paragraphs

Ability to create sets of documents, thesaurus terms, etc

Search over everything seamlesslySearch over everything seamlesslyIncluding annotations, tags, on everything

Including different kinds of document groups

» Associated with this office action

» Associated with this patent family

19

What Examiners Want: Improved Search

FastFastMost important feature for all aspects of PE2E

NPL integratedNPL integratedOnly write the query once, in one placeOnly one query language to learnResults are integrated with patent search results

References immediately accessibleReferences immediately accessibleIncluding foreign patentsIncluding booksFast translations, better translations (in some cases)

20

What Examiners Want: Improved Search

Use relevant contextUse relevant context– Show queries I’ve used, others have used in this situation– Show documents I’ve used, others have used in this

situation» Use these as a starting point, and expand out

Intelligent search as an optionIntelligent search as an option– Have suggested art ready when the case is opened– Allow selection of a piece of text (claim, etc) and

automatically search for related content– Improved search over specialized information (ranges,

equations, drawings)– Show highlighted, relevant snippets from the full text in the

search results

21

Social SearchSocial Search Examiners can benefit from a system that Examiners can benefit from a system that

lets them leverage the knowledge and lets them leverage the knowledge and work of otherswork of others

This may be where the new system This may be where the new system innovates mostinnovates most

22

User Research Results:User Research Results:Goal SurveyGoal Survey

23

Comments on the Comments on the Director’s BlogDirector’s Blog

24

Comments on the Comments on the Director’s BlogDirector’s Blog

25

Search SurveySearch Survey

26


27


28


29


30


31


32


33


34


35


36


37


38


39

How Can Researchers Help?How Can Researchers Help? Relevant evaluations of current tools andRelevant evaluations of current tools and Inventing new toolsInventing new tools

For comparing fragments of textFor precisely finding documents that are

similar, taking inventive concept into account

For doing image similarity comparisonsFor synonym expansion

40

Three Initial UI DesignsThree Initial UI Designs Posted a solicitation for UI design firms Posted a solicitation for UI design firms

summarizing the user research so far.summarizing the user research so far. Received initial designs in response, for Received initial designs in response, for

which we are selecting a set to evaluate.which we are selecting a set to evaluate.

43

Guided Query ConstructionGuided Query Construction

48

Thank you!Thank you!

1 User Research for Improving Patent Examination Tools Marti Hearst Chief IT Strategist, USPTO (on...

Documents

Transcript of 1 User Research for Improving Patent Examination Tools Marti Hearst Chief IT Strategist, USPTO (on...