8/14/2019 CIapps

1/14

1

Collective Intelligence Applications

Dawn G. Gregg*

dawn.gregg@cudenver.edu

A collective intelligence application is one that harnesses the knowledge and work of its

users to provide the data for the application and to improve its usefulness. The most hyped

examples of collective intelligence applications have been labeled as Web 2.0 applications.

Web 2.0 is an amorphous term used to define a computing paradigm that uses the Web as the

application platform and facilitates collaboration and information sharing between users [7].

Classic examples of Web 2.0 applications include: wikis, blogs (or weblogs), social network

services, and social bookmarking [2]. Web 2.0 sites are database-driven and are considered to be

infoware, in that, they are data intensive and the more data they contain the more valuable they

become [6].

Collective intelligence is a fundamentally different way of viewing how applications can

support human interaction and decision making. Most pre-Web 2.0 applications have focused in

improving the productivity or decision making of the individual user. The emphasis has been on

providing the tools and data necessary to fulfill a specific job function. Under the collective

intelligence paradigm, the focus is on harnessing the intelligence of groups of people to enable

greater productivity and better decisions than are possible by individuals working in isolation.

This suggests that software developers need to have different ways of thinking about how

their how software might be used and what features would enable better visualization and use of

information among groups of people. The new breed of collective intelligence applications

needs to center around user defined data that can be reused to support decision making, team

*Correspondence should be addressed to: Dawn Gregg, University of Colorado Denver and Health Sciences Center,

The Business School, Campus Box 165, P.O. Box 173364, Denver, CO, 80217-3364

8/14/2019 CIapps

2/14

8/14/2019 CIapps

3/14

3

Figure 1: DDTrac Architecture

The special education domain was selected for this study for three reasons.

First, special education students often have many people involved in their education andtherapy team. Students can receive services in school, at home, in clinical outpatient

environments and from consultants from other regions of the country. These

practitioners rarely have time to communicate details related to student progress which

can lead to an inefficient duplication of effort, gaps in treatment, or team members

working towards conflicting goals.

Second, special education literature emphasizes the importance of using data collectionand analysis procedures to monitor academic, social and behavior progress of students

with intensive special education needs [e.g. 1, 4, 5]. However, currently there are no

applications that support the data collection and analysis needs of this sector.

Third, there is the opportunity to harness the collective intelligence of these practitioners

8/14/2019 CIapps

4/14

4

to identify patterns of behavior and best practices, for both individual students as well as

groups of students with similar disabilities.

This suggests a need for software applications that can simplify the data collection and analysis

activities of special education practitioners and harness their collective intelligence to improve

their ability to make decisions about the children they serve.

The DDTrac system serves two primary purposes. First it serves as a communication

medium for therapists and teachers so that they know what to do when they sit down to work

with the special needs child. Second it collects data and provides data analysis tools to enhance

the ability to assess the adequacy of student progress and determine whether and when

instructional adjustments are necessary. The DDTrac system, shown in Figure 1, consists of four

functional areas:

The data entry section allows four different types of data entry: instructionalobjective and target data, observed social data, behavior data and narrative comments.

Thegoals and objectives section allows the creation and maintenance of long-termgoals tailored to the needs of the individual student and shorter term objectives that

define the activities involved in the day to day treatment and education of the child.

The data analysis section includes reporting and charting. These features make it easyfor special education teachers and therapists to examine student progress and modify

student's objectives and targets to maximize a student's learning outcomes.

The administration section contains additional functionality that was included to meetother identified needs (e.g. to manage access to data).

Distributed Data Entry. Special education instruction frequently occurs in locations

throughout a school (e.g. the special education classroom, specialized therapy rooms, the regular

8/14/2019 CIapps

5/14

5

education classroom, in the gym or on the playground) and can also occur at home and in the

community. Similar to traditional Web 2.0 applications, DDTrac uses the Web as the application

platform which allows data entry forms customized to the individual student to be accessed from

anywhere there is an internet connection. DDTrac users can choose to take data in two different

ways: either online using a wired PC or any wireless device OR offline, using a downloadable

web form which can be uploaded later when a network is available. Figure 2 shows a series of

data collection screens as would be displayed on a standard PC.

Figure 2: DDTrac Instructional Data Entry1

Another way applications can support distributed data entry is by supporting a wide

variety of data input devices. Using the web as a delivery platform is one way to support

multiple platforms. However, to support handheld devices in addition to traditional PCs,

DDTrac also uses style sheets targeted at different devices (e.g. mobile devices) and has data

1 All screenshots shown in this paper are for a fictional student Daniel Doo

8/14/2019 CIapps

6/14

6

entry screens designed for devices as small as 320 pixels wide. In addition, the quantitative data

entry is all accomplished using standard HTML form elements and were optimized for use with a

stylus, the common data entry tool on many handheld devices.

If the user does not have access to the Internet OR if the user has a poor/slow Internet

connection they can also use offline data entry. In offline data entry mode all of the data entry

forms for the selected students are downloaded into a single long page that has links to allow the

user to quickly navigate to specific locations within the page. The data entered in the offline data

entry page is automatically saved to cookies every 2 minutes and remains saved until the data is

successfully uploaded. If the user accidentally closes the page they just redownload the offline

data entry page and the page will refill with the data stored in the cookie.

Student-Centric Blog. Qualitative data is an important part of the information exchanged

in many collective intelligence environments. For example, special education teachers use back

and forth books to communicate daily notes to parents and parents write back to teachers about

events at home that might impact learning in the special education classroom. In Applied

Behavior Analysis2

(ABA) programs daily communication needs are met through handwritten

daily notes to parents and other practitioners.

Qualitative data in special education programs is frequently the only way to capture the

complexity and the transactional interaction between the setting and the students performance or

behavior [8]. DDTrac allows the capture of observation notes related to instructional activities,

during social interactions and following behavior episodes. These observation notes are stored in

2 Applied Behavioral Analysis is an approach to teaching behaviors and cognitive skills to children with autism and

other developmental disabilities that uses careful monitoring and positive reinforcement or prompting to teach each

step of a skill. Data collection typically consists of a designation as to whether a response is correct, incorrect,correct but prompted or if no response was given. Qualitative notes are also taken to communicate major difficulties

or successes [5].

8/14/2019 CIapps

7/14

7

a student-centric blog that includes the date and time they were recorded, the name of the

practitioner making the comment, and the name of the student the comment is being made about.

The descriptive notes represent the practitioner's best efforts to record what is occurring in the

context of the therapy session (e.g. describe student mood and overall performance on tasks).

The qualitative observations can also include the practitioners interpretation of how or why

certain behaviors unfolded as they did [8].

DDTrac automatically displays the five most recent days of observations about a

particular student as soon as the practitioner selects a student to work with. These qualitative

notes are an important mechanism for communicating recent changes in the child and in the

childs educational programs between distributed team members.

Commenting. Similar to traditional blogs, the commenting feature allows DDTrac users

to comment on observations made by others. The comments are attached to a particular student-

centric blog post and allow users to share experiences and provide suggestions related to issues

raised about a student. It allows for a dialog between users so that the approaches that work best

with a given student can be identified and adopted by the entire education team. The ability to

comment and share insights is critical to DDTrac's support of the collective intelligence of

special education groups.

Tagging. In paper-based special education data collection environments much of the

qualitative data found in the daily comments are lost within days of capturing it. The volume of

data generated in the special education programs of developmentally disabled children can be

overwhelming. Performing any meaningful analysis of this qualitative data (which can be

accumulated over periods of years) is virtually impossible.

Tagging is one mechanism that can be used in collective intelligence applications to

8/14/2019 CIapps

8/14

8

improve the usefulness of both qualitative and quantitative data. For example, the DDTrac

system supports different types of tagging for different types of data. Every time a practitioner

works on a particular objective with a student they can tag the entire work activity to describe

the childs overall mood during the activity. These predefined tags are designed to allow

practitioners to quickly convey the students mood to others and allow later assessment of the

impact of mood on the students performance.

Frequently special education students with intensive needs also have associated

emotional and behavioral disorders [4]. Behavior data documents a student's patterns of

behavior and is used to determine if efforts to minimize problem behaviors are effective.

Behavior data collection in DDTrac includes a variety of quantitative data related to the behavior

including the date, time, duration and number of behaviors counted during a behavior episode. It

also allows the practitioner to define a set of tags to allow problem behaviors to be analyzed in a

variety of ways. The practitioner defines tags for the types of behaviors being tracked for the

student, the trigger that preceded the behavior, and the location where the behavior occurred.

These tags can then be used to better understand patterns of behavior for a single student or for

groups of students.

For example, behavior tags can be used to generate a stacked bar chart (see Figure 3)

showing how individual behaviors contribute to the number or duration of behaviors observed

for a student. This allows users to see how individual behaviors are changing over time and

determine if replacement behaviors are increasing and less desirable behaviors are decreasing.

8/14/2019 CIapps

9/14

9

Figure 3: Stacked Bar Behavior Chart

The final type of tagging available in DDTrac is the semantic tagging of the narrative

comments taken as a part of the daily instructional, behavior or socialization observation notes

taken by practitioners. These semantic tags closely resemble the tags common on many Web 2.0

sites (e.g. Flickr, Delicious, Blogger etc.). They are freely chosen keywords which allow for

overlapping associations and that can be used for later retrieval and analysis of specific

comments. For example, a student may exhibit a finger flicking behavior infrequently. The

practitioner might note this in the daily notes along with other observations. Then, if the

behavior becomes a problem, the practitioner could retrieve all of the comments tagged

flicking to look for any patterns.

Goals & Objectives Wiki. The education programs of developmentally disabled children

are defined in an Individual Education Program (IEP), which establishes long-term goals and

short-term objectives tailored to the needs of the individual student [9]. The IEP also includes

descriptions of the students current level of performance, strengths, and individual needs. In

most schools this document includes input from several different people including the special

education teacher, the regular education teacher, the therapy specialists, the students parents and

external advocates. The IEP is an important document because it defines the direction for

treatment to be taken for the upcoming year.

8/14/2019 CIapps

10/14

10

DDTrac includes a goals and objectives wiki to meet this need. The wiki utilizes a

template that contains interconnected areas for goal creation, current level of performance

discussions, strengths, and greatest needs. Using a wiki structure for creating IEPs allows

practitioners with appropriate access permissions to add, edit, and delete goals as the new IEP

evolves. It is also possible for current IEPs to reference past IEPs for the same student, as well

as district standards or tests that the student should meet during the year.

The ease with which wiki pages can be created and updated is essential for the success of

the collaborative IEP generation tool. In addition, the ability to review changes before they are

added to the document and to roll back changes that dont meet the approval of the teacher or

parents is essential in an environment where the IEP represents a legal contract between the

school and the parents. The goals & objectives wiki allows IEP goals and objectives to be edited

until the IEP is accepted then the approved goals and objectives are added to the data collection

portion of DDTrac so data collection can begin.

Support Wiki and Blog. Two other collective intelligence features were designed into the

DDTrac application. A wiki is being used for all software documentation and a blog is used to

communicate with users. Both are essential for feeding the collective intelligence of users back

into the software. The documentation wiki allows developers to quickly add documentation

related to new features. In addition, it allows users to edit the documentation themselves to

clarify instructions that are not clear or to add documentation that the developers did not think to

create. The blog allows the developers to post announcements about new features that have been

added (essential in an environment where updates go live every two to three weeks) and solicits

feedback from users on which new features they feel will be most beneficial.

8/14/2019 CIapps

11/14

11

Collective Intelligence in Practice

DDTrac was deployed in an eighteen month field trial with one student with autism. The

student participated in speech therapy, occupational therapy, ABA therapy, and socialization

therapy known as Relationship Development Intervention in a home environment as well as

receiving special education services at school. The practitioners working with the student rarely

met in person and instead used the web-based DDTrac application for data collection and

communication. Over the course of the eighteen month trial data was captured for 481 separate

work sessions and included more than 50,000 individual pieces of data. The wiki and the blog

were active for the final 6 months of the project. During the 6 months the wiki had 163 pages

added and the blog had 33 posts documenting 12 software upgrades.

All participants in the field study felt that DDTrac significantly enhanced their ability to

take data and evaluate the performance of the student they were working with. They reported the

following benefits to using DDTrac:

1. The distributed data collection features made data collection easier and faster.2. The student centric blog enabled them to quickly understand the students recent behavior

trends and better prepare for their own work sessions.

3. The goal & objective wiki enabled them to easily develop new goals and objectives aswell as better understand the goals and objectives the student was currently working on.

4. The ability to analyze the data in a wide variety of ways enhanced their ability to assessstudent progress and made it easier to comply with mandatory reporting requirements.

5. The support blog and the wiki enabled them to learn more about DDTrac so that theycould use it more effectively.

The biggest benefit reported by the students parents was the ability to analyze long-term

8/14/2019 CIapps

12/14

12

education and behavior patterns. The parents commented that they had implemented numerous

interventions with their child and the data provided by DDTrac enabled them to have an

unbiased measure of whether or not a particular intervention had an impact either on the

educational outcomes or on the behaviors of their child.

Conclusion

This paper illustrates the benefits collective intelligence applications can provide in

specialized domains. The DDTrac collective intelligence application allows special education

data to be captured and shared more efficiently than the pencil and paper methods currently

being used (Figure 4). In addition, its reporting and charting features allow this data to be

analyzed comprehensively and quickly. This allows practitioners to spend more time working

with their students. It will also help to provide more efficient treatment and education plans, and

improve the outcomes of millions of children and adults with cognitive impairments.

Figure 4: Replacing paper data collection with distributed data collection and data analysis

Use of a specialized collective intelligence application, like DDTrac, can potentially benefit

organizations in a wide variety of domains (e.g. health care, outsourcing environments). The

ability to apply the collective intelligence of individuals working on similar problems is an area

that has just begun to be addressed by software developers; however, these systems will change

the way information is shared and used and has the potential to dramatically improve decision

making.

8/14/2019 CIapps

13/14

13

References

1. Deno, S.L. "Developments in Curriculum-Based Measurement," The Journal of SpecialEducation37, 3 (March 2003) 184-192.

2. Gibson, S. "Wikis are alive and Kicking in the Enterprise" eWeek.com, (20 Nov. 2006) (Accessed on September 10,2007).

3. Gregg, D. "Developing a Collective Intelligence Application for Special Education,"Working paper, (September 2007) (Accessed on September 10,

2007).

4. Gunter, P. L., Callicott, K., Denny, R. K., and Gerber, B. L. Finding a place for DataCollection in Classrooms for Students with Emotional Behavioral Disorders,Preventing

School Failure, 47, 1 (Fall 2003), 4-8.

5. Lovaas, O. I. Behavioral treatment and normal educational and intellectual functioning inyoung autistic children,Journal of Consulting and Clinical Psychology, 55, 1 (February

1987), 3-9.

6. McFedries, P. "The Web, Take Two,"IEEE Spectrum, (June 2006), 68.7. O'Reilly, T. What Is Web 2.0: Design Patterns and Business Models for the Next

Generation of Software, O'Reilly Media, Inc., (30 Sept. 2005),

(Accessed on July 9, 2007).

8. Schwartz, I. S. and Olswang, L. B. Evaluating child behavior change in natural settings:Exploring alternative strategies for data collection, Topics in Early Childhood Special

Education, 16; 1 (Spring 1996), 82-101.

9. Wright, Peter and Wright, Pam IEP Goals & Objectives: A Tactics and Strategy Session(2007) (Accessed

on August 31, 2007).

8/14/2019 CIapps

14/14

14

Collective Intelligence Sidebar

Collective Intelligence Application Requirements (adapted from 3, 7)

1. Task specific representations: Domain specific collective intelligence applicationsshould support views of the task that are tailored to the particular domain.

2. Data is the key: Collective intelligence applications are data centric and should bedesigned to collect and share data among users.

3. Users Add Value: Users of collective intelligence applications know the most about thevalue of the information it contains. The application should provide mechanisms for

them to add to modify or otherwise enhance the data to improve its usefulness.

4. Facilitate Data Aggregation: The ability to aggregate data adds value. Collectiveintelligence applications should be designed such that data aggregation occurs naturallythrough regular use.

5. Facilitate Data Access: The data in collective intelligence applications can have usebeyond the boundaries of the application. Collective intelligence applications should

offer web services interfaces and other mechanisms to facilitate the re-use of data.6. Facilitate Access for All Devices: The PC is no longer the only access device forinternet applications. Collective intelligence applications need to be designed to integrateservices across handheld devices, PCs, and internet servers.

7. The Perpetual Beta: Collective intelligence applications are ongoing services providedto its users thus new features should be added on a regular basis based on the changing

needs of the user community.