IEICE TRANS. INF. & SYST., VOL.E94–D, NO.3 MARCH 20111

PAPER Special Issue on Knowledge Discovery, Data Mining and Creativity Support System

GKJ: Group KJ Method Support System Utilizing Digital Pens

Motoki MIURA †a), Member, Taro SUGIHARA ††b), Nonmember, and Susumu KUNIFUJI ††c), Member

SUMMARYTypically, practitioners of the KJ method use paper labels and four-

colored ball-point pens to externalize their thoughts and ideas during a cre-ative thinking process. A similar approach and method is used in groupKJ lessons. However, due to the large paper size required, this approachis limited in the effective capturing and sharing of diagrams. Consideringthe merits of the conventional paper-pen approach and the demand for aquick sharing of diagrams after the session, we designed and implementeda system to digitize the group KJ session—not just the diagrams but alsothe details of the creative work processes. We used digital pens during thesession to capture the position and orientation of labels as well as their con-tents. We confirmed the efficiency of our system over the course of severalKJ sessions.key words: CSCW, Creative meeting, Label work, KJ method

1. Introduction

We often use small paper cards or Post-it notes to organizeour thoughts and ideas. The cards are generally arranged ororganized on a table or adhered to a wall. The KJ methodproposed by Jiro Kawakita [1]∗ is one of the technologiesutilizing this method.

The organization of paper cards has two advantages:(1) moving the cards by hands is intuitive, and (2) groups ofusers can simultaneously access the cards. Therefore, suchcard-based activity is commonly used for summarizing andorganizing ideas. However, archiving the creative thinkingprocess and outcome (usually an affinity diagram made ofcards) and sharing them with the participants are difficultbecause of the large working space. Digitizing the processof card-based activities must address special concerns. Ifwe could archive and share the process and outcome easily,it could help to continuously improve and refine the ideas.

Several tools can handle virtual cards and Post-it noteson a PC, such as the KJ Editor[2], GUNGEN[3] and D-ABDUCTOR[4]. These tools are effective in digitizing thecard-organizing task. Also, they can solve the issue ofarchiving and sharing the process and its outcome. How-ever, using mouse and keyboard input is not suitable forgroup work. The intuitiveness of card handling can also be

Manuscript received February 25, 2010.Manuscript revised 0, 2010.Final manuscript received 0, 0.

†Faculty of Engineering, Kyushu Institute of Technology††School of Knowledge Science, Japan Advanced Institute of

Science and Technologya) E-mail: miuramo@mns.kyutech.ac.jpb) E-mail: sugihara@jaist.ac.jpc) E-mail: kuni@jaist.ac.jp∗KJ method is a trademark of Kawakita Research Institute.

dismissed.To provide a digitized outcome of card-based activity,

one of the participants must digitize the large amount ofdata. This places an additional burden on the participant,and the assembly of the data is very time consuming.

To solve these issues, by introducing a natural and aug-mented reality approach, Klemmer et al.[5] proposed theDesigner’s Outpost system to capture the flow and transi-tion of Post-it notes on a wall-size large screen. We alsodeveloped a system to relieve the occlusion problem of theoutpost system using a glass surface with controllable trans-parency[6]. However, the outpost system’s large size makesit difficult to move, rendering it inapplicable for digitiz-ing activities in casual meetings that are normally held inknowledge-creation companies and research laboratories.

In this paper, we propose a method for capturing thelocation of the paper cards using a small and simple facility.The captured location is then used to reproduce the card-organization process after the session ends. We have devel-oped a system based on this method to easily archive theprocess and its outcome. We confirmed the effectiveness ofthe method, as well as the system, over the course of severalKJ-method sessions.

2. Proposed Method

Fig. 1 Digital pen and cards

Our method utilizes Anoto-based pens (Figure 1) toarchive card-based activities. The Anoto-based pen can cap-ture drawings on paper cards and a base sheet. The position

2IEICE TRANS. INF. & SYST., VOL.E94–D, NO.3 MARCH 2011

Fig. 2 Line over the card border is separated into three lines.

Fig. 3 If the three lines were drawn at almost the same time, the cardwould be located as shown in the right-hand figure.

Fig. 4 Recognizing the card orientation

Fig. 5 Detection of overlapping state

of the drawings is recognized by scanning special dotted pat-terns on the paper. Because of the unique features of thepatterns, the method can distinguish between the data of thedrawings on the cards and those on the base sheet. In ad-dition to these characteristics, the drawings can be used tospecify the positions, orientations and groupings of the cardson the base sheet.

2.1 Principle

When the user draws a line that covers the sheet and a card(Figure 2 left), the pen recognizes the line as three drawings(Figure 2 right). If these drawings are generated at almostthe same time, we can consider that, at that time, the papercard was placed so as to connect the three drawings (Figure3). We call this operationscanningand the connecting-pointjoint. If one joint is detected, the position of the card can bedetermined. If two or more joints are detected, the methodcan determine not only the position but also the orientationof the card (Figure 4). By enhancing this technique, themethod can recognize the overlapping state of multiple pa-per cards if two joints are extracted for each card (Figure5). To eliminate unnecessary pen markings on the paper forscanning, we can simply use a semi-transparent plastic sheetwhile scanning. The Anoto-based pen can scan dot patternsof the paper when such a transparent plastic sheet is used.

Fig. 6 Scanning for grouping and ungrouping.

Table 1 Processing of drawings in our method

Drawings on card Handled as normal drawings on thecard.

Drawings on base sheet Handled as normal drawings on thebase sheet.

Drawings on both cardand base sheet

Handled as the determining positionand orientation of the card.

Drawings on multiplecards (round stroke)

Handled as grouping operations.

Drawings on multiplecards (single stroke)

Handled as ungrouping operations.

A similar method was proposed by [7] as research ondigitized experiment-record notes and digital paper book-mark tagging in the CoScribe system[8]. However, to editthe card structure, we introduced extra pen gestures calledgrouping and ungrouping. Grouping can be performed by acontinuous round stroke from the top-level card to the childcards (Figure 6 left). This operation is derived from the ’sur-rounding of the cards’ metaphor. Ungrouping can be per-formed by a continuous single stroke from the top-level cardto the child cards (Figure 6 right). This operation is derivedfrom the ’sweeping of the cards’ metaphor. These groupingand ungrouping operations can be used for common card-based creative activities, especially for making affinity dia-grams using the KJ method[1]. The strategies of handlingdrawings and gestures on cards and base sheets using thismethod are summarized in Table 1.

A digital camera can also record the paper sheet im-ages in detail. However, the reusability of the card contentis crucial for creative tasks, and the atomic data should beprovided to enhance the process. In particular, an authenticKJ method involves procedures that utilize repetitive tasksfor refinement and deepening.

2.2 Merit

Here we describe the merits of the proposed method:

1. Without using PCs, the participants can edit theaffinity-diagram cards with digital pens and transpar-ent sheets. These devices are more natural and intuitivethan PC operations. Also, a digital pen is inexpensivecompared to a PC.

2. The proposed method is suitable for card-based cre-ative thinking. Most of the participants experienced

MIURA et al.: GKJ: GROUP KJ METHOD SUPPORT SYSTEM UTILIZING DIGITAL PENS3

with the KJ method are accustomed to handling cards.Therefore, the method does not disturb the concentra-tion of the participants.

3. The participants can contribute significantly to the ses-sion by expressing ideas as well as by controlling thecard position.

4. The archived diagram manages the cards separately, sothe diagram can be easily revised. Still images takenby a digital camera can capture the diagram; however,they are static.

5. Because of the wireless transmission of the digital pen,the method accepts simultaneous activities by multipleparticipants.

6. The Designer’s Outpost method [5] does not covermoving multiple cards at once. On the other hand, ourmethod can distinguish multiple moving cards. Themechanism of our method is simpler than other ARsystems. Our method exhibits a huge advantage to sys-tem mobility because of the small, lightweight devicesit employs.

3. Target activity and diagram

The proposed method mainly focuses on creating affinitydiagrams of using authentic KJ method in a group. In thissection, we describe details of the conventional activity, thediagrams and methods for readers’ benefit.

3.1 Discussion of the radial affinity diagram

Figure 7 shows a radial affinity diagram. The diagram is cre-ated to achieve rough consensus and understanding amongthe participants in the discussion. First, a theme or a keyproblem is placed at the center of the base sheet. Secondly,participants position their cards in turns, align similar cardsin a spoke fashion. These cards are related by links. Duringthe conventional process, a paper clip is often used to ex-press a temporary link. Finally, the participants fix the cardpositions and links. Occasionally, they select representativecards to render the following tasks efficient.

Fig. 7 Radial affinity diagram

3.2 Grouping cards

Figure 8 shows a snapshot of grouping cards. Firstly, theparticipants randomly arrange all cards on a grid. Secondly,they read through the cards and pick up two cards that are’relatively similar’. The picked cards are replaced close toeach other. After the pairing, the participants add a titlecard for each pair. A suitable title conveys the meaningsand essence of the paired cards, without being too abstract.The title cards and the two paired cards are placed in a pileand clipped. Next, the three cards should be handled as anew card that is labelled with the title. The process is re-peated until the total cards in the piles reach five to nine innumber.

Fig. 8 Grouping cards

3.3 Creating an affinity diagram

After grouping the cards, the participants unpack the piledcards and place them on the base sheet. Note that thehierarchical structure of the cards must be strictly pre-served. As shown in Figure 9, the cards are encircled withred/blue/green/black colored circles. Finally, the partici-pants provide a meaningful phrase or sentence to indicatethe top-level groups.

Fig. 9 Affinity diagram

4IEICE TRANS. INF. & SYST., VOL.E94–D, NO.3 MARCH 2011

4. GKJ system

With the proposed method, we have developed GKJ, a sys-tem to facilitate the archiving of card-based activity such asthe KJ method, in groups. GKJ accepts the scanning anddrawing by an Anoto-based pen on paper cards and imme-diately updates the status of virtual cards, which correspondto the real cards. The status (position and orientation) ofthe virtual cards is recorded with a timestamp to archive theprocess of the activity. With the archive, the sharing of theprocess can be enhanced once the meeting session is com-pleted.

The GKJ system consists of (1) Anoto pens, (2) an L-Box Digital Pen Gateway System (DPGW) and (3) a GKJeditor. The system overview is shown in Figure 10. The L-Box DPGW collects pen data from multiple pens simultane-ously via a Bluetooth connection and sends it to a MySQLtable in a PC. The GKJ editor checks the updated data andhandles the data to construct a digital representation of thecurrent paperwork status.

Fig. 10 GKJ system overview and data flow

The GKJ editor accepts inputs from a mouse and key-board for further editing tasks on the virtual cards. In thismanner, the participants can continuously review the pro-cess and refine it by editing the virtual cards. GKJ also pro-vides a function to print out the virtual cards for the preced-ing session.

Figure 11 shows the initial screen of the GKJ. InFigure 11, we assume five participants. Therefore, fiveworkspaces are presented. Each workspace consists of basesheets (gray rectangles) and cards (yellow rectangles). Thelarger number (01-05) indicates a workspace-ID. Basically,the workspace displays pen data from the same pen-ID.Therefore, the workspace-ID indicates the participants. Thesmaller number (01) shows a transfer-ID, which representsan additional destination of the pen data. If the transfer-ID differs from the workspace-ID, the pen data are placedin both workspaces. In the example of Figure 11, all pendata with pen-ID (01-05) are sent to workspace 01. Thismeans that participants (01-05) work together on workspace01 with the assembled pen data. If the transfer-ID is equal

to the workspace-ID, it means that the participants work asan individual. We included this feature, because in a groupKJ method session, group and personal work styles are of-ten changed to improve efficiency and initiative among theparticipants. When the work style changes from group to in-dividual, the participants will require personal copies of thecards. Using the printing function of GKJ, they can easilyobtain copies. The session organizer can confirm the statusof the individual tasks by browsing, as shown in (Figure 11).

4.1 Typical work style with Anoto-based pens

The following steps are the typical work style of the GKJsession:

1. The organizer distributes blank cards, on which the par-ticipants write their ideas. When they send the pendata, the GKJ workspace displays the written ideas incard areas.

2. The participants create a radial affinity diagram, as de-scribed in 3.1). The radial affinity diagram consists ofnodes (cards) and links. The participants put their cardson the base sheet in turns. If necessary, the organizerasks the participants to confirm their meaning and con-trols the turns. Each participant scans each card afterits position is fixed and writes additional cards inspiredby the contents of other cards. The GKJ workspace dis-plays the updated status of the radial affinity diagram.

3. When the radial affinity diagram is created, the par-ticipants can share their understanding of the cards.Then the participants select some representative cardsby picking them up from the radial affinity diagram.

4. The participants find pairs of cards (see Groupingcards, described in 3.2). Before pairing them, the par-ticipants are expected to scan all representative cards.After the initial scanning, the cards are scanned whilepairing, if necessary.

5. The participants create title cards by writing on a newblank card. Then they align the title cards and pairedcards and group them with circular pen gestures. Afterthe grouping operation, the cards are placed in piles andclipped as usual. The GKJ system displays the piles ofcards, as shown in Figure 12. The process is repeatedas per the conventional KJ method.

6. After the cards are grouped, the participants move tothe unpacking phase (see Creating an Affinity Dia-gram, described in 3.3). The former KJ method re-quired an additional marking on the label to preservethe hierarchical structure of the card. Moreover, theparticipants were required to estimate the necessaryarea for arranging all the unpacked cards in a layout.If the estimation failed, participants had to carefullymove many cards. The GKJ system eliminates takingcare of the layout, because the participants can simu-late the unpacking of the cards on a PC screen. Afterchecking the layout of the virtual cards, they can placethe cards on the physical base sheet.

MIURA et al.: GKJ: GROUP KJ METHOD SUPPORT SYSTEM UTILIZING DIGITAL PENS5

Fig. 11 Initial screen

Fig. 13 Final state of an affinity diagram

4.2 Extra functions

GKJ usually handles label text on a card as drawings, butit can be converted to typed text utilizing a handwriting-recognition engine. We already linked the GKJ system witha recognition engine and tested the effectiveness of the au-tomatic recognition. Figure 13 shows an example of a finalaffinity diagram with the typed text. The surrounding cir-

cular lines are interactively calculated from the virtual cardpositions. Therefore, the participants can modify the affinitydiagram without an additional burden. The diagram can beprinted or exported to PDF files using the iText PDF library.

All scanning and grouping actions on the cards arestored as history data. Thus, the participants can review theprocess by specifying a time in the past. The resumption ofediting from a previous state is also supported.

As described above, the GKJ system allows partici-

6IEICE TRANS. INF. & SYST., VOL.E94–D, NO.3 MARCH 2011

Fig. 12 Piled cards in the GKJ

pants to freely choose a proper environment (real or digi-tized cards) for their tasks including review of the data usingthe digitized log rollback and the distribution of data. Thehigh level of portability of the GKJ system makes it usefulin a variety of environments. Group sessions with the GKJsystem can be held with (1) a base paper sheet, (2) papercards, (3) digital pens, (4) L-Box, a small Linux box and (5)a PC.

Fig. 14 Practice session at JAIST (1)

Fig. 15 Practice session at JAIST (2)

5. Practice

We operated the GKJ system in small courses of collabo-rative card-base activity (Figure 14 and Figure 15 show a

Fig. 16 Practice session with city-hall staff (1)

Fig. 17 Practice session with city-hall staff (2)

class at our institute, and Figure 16 and Figure 17 show alecture course with city-hall staff). For the class at our in-stitute, we used pre-printed cards as materials, and the par-ticipants created a radial affinity diagram by positioning thecards spatially to represent their thoughts and ideas. In thiscase, the system may not have enhanced the ongoing study,but the participants enjoyed scanning and checking the dig-itized data. The instructor conducted the course in the samemanner as with conventional courses using cards. Figure 14shows a scene with the instructor holding up a card to showit to the participants. This was a significant behaviour whenusing paper cards. Because the scanning was intuitive andthe checking of the scanning result on GKJ was interesting,the instructor often captured the card after moving it.

In the lecture course at a city hall, the participants firstwrote their ideas and work-related problems on the cards.Then they created a radial affinity diagram by classifyingthe cards during the discussion. After arranging the cards,several participants scanned all the card positions.

In both courses, the participants acted naturally, asusual. They communicated implicitly by observing the be-haviour of other participants, such as placing a card on asheet. After both courses, a PDF file of the digitized radialaffinity diagram was sent to the participants. In a formercourse, it usually took the instructor several hours to dig-itize the diagram for sharing. Also, a precise card-history

MIURA et al.: GKJ: GROUP KJ METHOD SUPPORT SYSTEM UTILIZING DIGITAL PENS7

log was helpful for reviewing the contents of the session.We obtained the following findings from observations

of the sessions and comments from the participants.

Advantages

1. Card writing with a pen was straightforward and intu-itive for the participants.

2. The participants naturally organized their cards, be-cause they could observe other participants’ behaviour.

3. The quick distribution of digitized logs and figures(PDF) was effective for reviewing the session and dis-cussion.

4. The instructor and some of the participants masteredscanning and enjoyed the process.

5. The position of the scanned data accurately representedthe physical figures.

Drawbacks

1. Occasionally, the scanning failed due to errors. Themost frequent mistake was a scanning section error.The GKJ system utilizes four A2-size sheets to com-pose an A0-sized base sheet. Because the printed-dotpattern of the A2 sheet was the same, the user neededto specify the section of the base sheet to the system be-fore scanning by selecting a check box. Occasionally,the user failed to preset the scanner or scanned witha wrong pen. To reduce the likelihood of occurrenceof this error, we now prepare four preset pens for eachA2-sheet section. Even if the error occurs, the user caneasily fix the misrecognition by rescanning.

2. Occasionally, unnecessary scanning lines appeared oncards and base sheets because of misrecognition dur-ing scanning. The misrecognition was caused by weakpen pressure, high scanning speed and lack of a gap be-tween the cards (less than 1 cm). To solve this issue, weadded a ’transparent’ pen mode, which does not drawunnecessary lines while scanning.

3. Some participants wrote upside-down on the cards, be-cause it was difficult to recognize the top and bottomof the cards. This caused the card content to be shownwrong-side-up, and the position was scanned incor-rectly. This issue could be solved by implementing afunction to automatically detect when the wrong sidewas up by analyzing the handwritten note and handlingthe card carefully.

4. As we described in points 1 to 3, scanning requiredskill and know-how. The user needed to understandthe characteristics of the pen and GKJ to adequatelyoperate the system. However, this skill could be easilyacquired with a few minutes of training, and a failedscanning could be easily recovered by rescanning.

Even with its drawbacks, the GKJ system can augmentconventional paper-based discussion. Also, most of thedrawbacks could be solved by further system refinements.

6. Conclusion

We proposed a method for capturing the location and hier-archical structure of paper cards written with Anoto-basedpens. The method enables the participants to record a pre-cise, atomic, transition log of the cards. We also developed asystem for digitizing paper-based card organization tasks in-stantly, based on the proposed method. Due to the simplicityof the pen-based input, the GKJ system is universal; it canbe used by office workers and also by the elderly and pri-mary school children. We confirmed the effectiveness of thesystem during the course of several sessions. We applied itto small-group learning sessions of up to 10 persons, but thissystem is applicable for many participants and groups, be-cause it can handle up to 40+ pens at the same time. We willrefine the GKJ system to improve its usability, and it shouldcontribute to the effectiveness of group discussions that in-clude various types of participants, such as town meetings.

Acknowledgement

Digital Pen Gateway System and related technologies arefrom NTT Comware Tokai Corporation. Our research ispartly supported by a grant-in-aid for Scientific Research(20680036, 20300046).

References

[1] J. Kawakita, The Original KJ Method (Revised Edition), KawakitaResearch Insitute, 1991.

[2] H. Ohiwa, N. Takeda, K. Kawai, and A. Shiomi, “Kj editor: a card-handling tool for creative work support,” Knowledge-Based Systems,vol.10, no.1, pp.43–50, 1997.

[3] J. Munemori, “Gungen: Groupware for a new idea generation supportsystem,” Inf. Soft. Technol., vol.38, no.3, pp.213–220, 1996.

[4] K. Misue, K. Nitta, K. Sugiyama, T. Koshiba, and R. Inder, “Enhanc-ing D-ABDUCTOR Towards a Diagrammatic User Interface Plat-form,” Proceedings of KES, pp.359–368, 1998.

[5] S.R. Klemmer, M.W. Newman, R. Farrell, M. Bilezikjian, and J.A.Landay, “The Designers’ Outpost: A Tangible Interface for Collabo-rative Web Site Design,” Proceedings of UIST’01, pp.1–10, 2001.

[6] M. Miura and S. Kunifuji, “A Tabletop Interface Using ControllableTransparency Glass for Collaborative Card-based Creative Activity,”Proceedings of KES2008, LNAI5178, pp.855–862, Sept. 2008.

[7] H. Ikeda, N. Furakawa, and K. Konoishi, “iJITinLab: InformationHandling Environment Enabling Integration of Paper and ElectronicDocuments,” CSCW2006 Workshop (Collaborating over Paper andDigital Documents), Nov. 2006.

[8] J. Steimle, O. Brdiczka, and M. Muhlhauser1, “Collaborative paper-based annotation of lecture slides,” Journal of Educational Technol-ogy and Society, vol.12, no.4, pp.125–137, 2009.

Motoki Miura was born in 1974. He re-ceived B. S., M. E. and D. E. degrees in elec-tronics engineering from University of Tsukuba,

8IEICE TRANS. INF. & SYST., VOL.E94–D, NO.3 MARCH 2011

in 1997, 1999 and 2001, respectively. FromAugust 2001 to March 2004, he worked asa research associate at TARA centre, Univer-sity of Tsukuba (2001-2004), assistant profes-sor in School of Knowledge Science, JapanAdvanced Institute of Science and Technology(2004-2009). He is currently working as an as-sistant professor in School of Knowledge Sci-

ence, Japan Advanced Institute of Science and Technology. He is a memberof IEICE, JSAI, IPSJ, JSSST, ACM, JSET and HIS.

Taro Sugihara was born in 1977. He grad-uated from the advanced course of TokuyamaCollege of Technology and received a B.S. de-gree in engineering from the National Institu-tion for Academic Degrees and University Eval-uation (NIAD-UE) in 2000 and M.E. and D.E.degrees in engineering from Kyoto Institute ofTechnology in 2002 and 2005, respectively. Heis currently working as an assistant professor inSchool of Knowledge Science, Japan AdvancedInstitute of Science and Technology. His re-

search interests include human-computer interaction, especially user be-haviour, and adult education. He is a member of ACM and the HumanInterface Society.

Susumu Kunifuji was born in 1947. Hereceived B. E., M. E. and D. E. degrees fromTokyo Institute of Technology in 1971, 1974and 1994, respectively. He worked as a re-searcher at International Institute for AdvancedStudy of Social Information Science, FUJITSULtd. (1974-1982), Chief researcher at the Insti-tute for New Generation Computer Technology(1982-1986), Manager of International Institutefor Advanced Study of Social Information Sci-ence, FUJITSU Ltd. (1986-1992), Professor of

School of Information Science at JAIST (1992-1998), Director of Centrefor Information Science at JAIST (1992-1998). He is currently a professorat School of Knowledge Science, Japan Advanced Institute of Science andTechnology. He is a member of IEICE, JSAI, IPSJ, SICE, JCS, etc.