NLM TEHP Final Report Solomon Solutions [email protected] Voice 201.434.0404 Mobile:...

NLM TEHP Final Report

Solomon [email protected]

Voice 201.434.0404Mobile: 917.601.1273

www.Solomon-Solutions.com©2006 Solomon Solutions

Qualitative Research Studyon

Environmental Health and Toxicology

Prepared for:

April 12, 2006

2

©2006 Solomon Solutions

I. Background

II. Study Objectives

III. Methodology: Online Focus Group

IV. Summary Conclusions

V. Overall Findings: Environmental Health & Toxicology

VI. Recommendations

VII. Appendix

3


Background

As part of an ongoing qualitative evaluation of NLM websites, the National Library of Medicine conducted TWO online focus groups on Environmental Health and Toxicology – a portal to databases and resources to help professionals and laypeople in the environmental health and toxicology community.

The primary objective of the study was to understand the value, usefulness, strengths, and weaknesses of the Environmental Health and Toxicology website for potential users of the site -- both professional and non-professional.

The online focus groups on the Environmental Health and Toxicology site were conducted on April 6, 2006.

4


I. Background





VI. Recommendations

VII. Appendix

5


Study Objectives

The main objective of this qualitative study on Environmental Health and Toxicology was to evaluate the value and usefulness to current and potential users. Additional objectives included:

Gauging the site's strengths and weaknesses, in terms of content, navigation, and overall structure

Understanding how users currently use and intend to use the site

Getting reactions to acronyms and other terminology used on the site

Target respondents. Screening parameters for respondents were as follows:

Mix of current and potential users

Group 1 was comprised of professionals employed in the field of environmental health and toxicology as physicians, academics (educators, researchers, etc.), nurses, and librarians.

Group 2 was comprised of concerned citizens -- those with a strong concern about chemicals in the environment and interest and/or participation in activities related to conservation and environmental health.

Group 1 was recruited from a listserv subscription within the environmental health and toxicology community; group 2 was screened and recruited via telephone using a recruiting agency.

All were asked to spend approximately 20 minutes viewing/navigating the site and completing a series of tasks prior to the discussion.

6


I. Background





VI. Recommendations

VII. Appendix

7


Methodology: Online Focus Group*

Testing consisted of TWO (2) 90-minute online focus groups.

Respondents were recruited via listserv subscription (Professionals) and telephone (Concerned Citizens).

Group 1 (Professionals) consisted of 11 participants, and Group 2 (Concerned Citizens) consisted of 12 participants. (See Appendix.)

The group was asked to visit the Environmental Health and Toxicology site prior to the session. They were also asked to complete a series of tasks using the site.

Participants in the "Professionals" group received an incentive payment of $45 (though two participants waived payment), and participants in the "Concerned Citizens" group received a $75 incentive payment.

* The online focus group represents a qualitative methodology used for the purposes of ideation, brainstorming, and evaluation. Qualitative methodologies are based on a small sample size, and the findings are intended to be directional only, not projectable to the larger population.

8


I. Background





VI. Recommendations

VII. Appendix

9


Summary Conclusions

Impressions of Environmental Health and Toxicology

Reactions to the Environmental Health and Toxicology website from both Professionals and Concerned Citizens were enthusiastically positive.

Respondents offered several constructive critiques of specific areas of the site, but overall, it was perceived as an extremely useful, handy, valuable tool for the environmental health and toxicology community -- for both the public and professional sectors.

Respondents liked the idea of having a central portal to use as a "starting point" for all of their searches on chemical toxicity and related topics.

Very few had difficulty navigating, though there were some areas that seemed overwhelming to respondents, in terms of the sheer amount of content.

Minor obstacles emerged as areas for improvement that could be addressed with tweaks to the site. For instance, there was some confusion as to the purpose of three search mechanisms on one screen -- for SIS, TOXNET, and NLM.

The use of acronyms presented confusion to some respondents -- particularly in the Citizens group (who were less familiar with what they represented) -- but this was seen as easily addressed with mouse rollovers or on-screen definitions.

10


Summary Conclusions (cont’d.)

Strengths and Weaknesses: Environmental Health and Toxicology

Strengths of the Environmental Health and Toxicology site were its logical and "clean" layout, ease of use, and comprehensive coverage of information.

Weaknesses of the site included the (perceived) excessive use of acronyms (without obvious definition) and the presence of multiple search tools.

Suggestions for change included expanding the FAQ section; adding categories such as "community volunteers," "parents," "kids," and "risk assessors" to the "Especially For" section; and adding on-screen definitions of the mouse rollovers for the acronyms.

Several suggestions emerged for expanding some areas (such as "Especially For," as mentioned above, and "Reference Tools") and shortening others (e.g., "More to Explore"), but overall, the site was perceived as extremely useful.

The concept of online tutorials or videos was met with a mixed reaction. Most liked the idea of online tutorials, provided there was no cost, but the idea of videos on-screen yielded concerns about download time and other associated factors (e.g., access speed).

11


Summary Conclusions (cont’d.)

Environmental Health and Toxicology

Respondents expect to use the Environmental Health and Toxicology site frequently, and in many cases have already referred colleagues and friends to the site.

Overwhelmingly, the issue of data reliability was high (trustworthy and reliable), though a select few in the Citizens group expressed concern that a site associated with the government may have a political bias or "policy-based" censorship.

Respondents would like to learn more about NLM resources via e-mail, conferences, local community organizations, and professional organizations.

In contrast to other resources in the field of environmental health and toxicology, however, the site was seen as head-and-shoulders outstanding.

Overall, the site does a great job of providing a comprehensive, inclusive starting point for both professionals and laypeople seeking in-depth information on environmental health, toxicology, and related topics.

Following are detailed findings from the testing on Environmental Health and Toxicology.

12


I. Background





VI. Recommendations

VII. Appendix

13


Overall Findings: Environmental Health and Toxicology

Current information gathering

Most respondents use the Internet to find information on environmental health and toxicology.

Current patterns of information-gathering on environmental health differ between the Professionals and the Concerned Citizens. Professionals reported going to NLM sites and professional organization sites (e.g., NIOSH, OSHA, EPA, WHO), while Citizens are more likely to search for information via Google or another Internet-based search engine.

Some also use the CDC, PubMed, and WebMD.

Users in the Professionals group reported that they characteristically seek information on health effects of chemicals, peer-reviewed studies and scholarly research, and toxicity data on household and other products.

Those in the Concerned Citizens group are more interested in a broad range of topics that impact their own situations or their daily lives. Examples included floor cleaners, medicines and prescriptions, clean water issues, automotive oil, radiation and chemotherapy.

14



Current information gathering (cont'd.)

Respondents in both groups also were asked if they seek information on specific topics.

The majority of professionals but few citizens said that they seek information on nitrogen compounds, solvents, gases/vapors, plastics and rubber. In the Citizens group, some pointed out that they had searched for information on hybrid cars and garage chemicals.

Most professionals reported that they do seek information on pesticides, but among Concerned Citizens, there was less interest in this topic.

Hazardous household products was reported as a common search topic among Concerned Citizens, and some in the Professionals group also seek information in this area.

"Occupational safety" was not high on the list of search topics in either group, but "drinking water," on the other hand, was an increasingly major search topic across the board.

15



Common Challenges in Gathering Information

The types of challenges commonly encountered in seeking environmental health and toxicity information involves recency of data, "wading through" the sheer breadth of information (which can be overwhelming, particularly to the layperson), and -- for both groups -- finding a "one-stop" site for this type of information.

For some, the specificity of the information made the information seemed largely geared toward the Professional user.

“The lay person's expression of toxicity and risk are very different from the professional community's use of those words. They are more broad, ours more

specific.” (Marcus, Professional)



“The databases are so specific that it's often a challenge to know which one to search for the exact data.” (Megan, Professional)

“The databases are so specific that it's often a challenge to know which one to search for the exact data.” (Megan, Professional)

“sometimes there’s too much info - where to start?” (Lori, Citizen) “sometimes there’s too much info - where to start?” (Lori, Citizen)

16



Common Challenges in Gathering Information (cont'd.)

Similarly, several in the Citizens group explained that the terminology is not fully accessible for the layperson -- that it tends to be "too technical" and requires "translation."

Additionally, for some in the Citizens group (possibly due to lower recognition of the sources), there was some concern as to the reliability of the information.





“Usually the info is either too general to be useful, or so technical that's it hard to interpret. If it's anything chemically related, I ask

my father the chemist!” (Shalom, Citizen)

“Usually the info is either too general to be useful, or so technical that's it hard to interpret. If it's anything chemically related, I ask

my father the chemist!” (Shalom, Citizen)

“reliability of info - can it be trusted?” (Laura, Citizen) “reliability of info - can it be trusted?” (Laura, Citizen)

“Challenges include sorting through the massive amount of literature. Who is right, wrong, solid research or a professor publishing something for tenure. I do

not know who I can trust if a private site.” (Mark, Citizen)

“Challenges include sorting through the massive amount of literature. Who is right, wrong, solid research or a professor publishing something for tenure. I do

not know who I can trust if a private site.” (Mark, Citizen)

“… sometimes the commercial websites have the most information, and the challenge … is to read through the lines because … they are trying to sell their products!” (Lorraine, Citizen)

“… sometimes the commercial websites have the most information, and the challenge … is to read through the lines because … they are trying to sell their products!” (Lorraine, Citizen)

17



Addressing these Challenges

Respondents reported that they typically address these challenges by limiting search terms, reorganizing the information so they can "digest" it, and checking with peers and colleagues.

On the other hand, those in the Citizens group reported addressing challenges by checking information for consistency against other sources and occasionally asking others.

“resolving challenges -- time, brainbusting, talking it out with respected peers, trying to find a new perspective/out of the box angle.” (Marcus, Professional)

“resolving challenges -- time, brainbusting, talking it out with respected peers, trying to find a new perspective/out of the box angle.” (Marcus, Professional)

“When searching toxline, for example, I use a lot of very specific limiting terms.” (Nancy, Professional)

“When searching toxline, for example, I use a lot of very specific limiting terms.” (Nancy, Professional)

18



Impressions of Environmental Health and Toxicology

All of the respondents in the Professionals group but few of those in the Concerned Citizens group had heard of or visited either Environmental Health and Toxicology OR TOXNET before.

Regardless of past familiarity or visitation, however, impressions were overwhelmingly positive.

Several respondents in the Professionals group had used the site for information on poisoning, chemical exposure levels, and chemical structures. Several were also familiar with TOXMAP and Tox Town.

"I had been there. I am familiar with a host of NLM sites and appreciated the similarities in layout.” (Megan, Professional)

"I had been there. I am familiar with a host of NLM sites and appreciated the similarities in layout.” (Megan, Professional)

"My overall impressions were very POSITIVE. I like the fact that this is like the encyclopedia for environmental health.” (Shyam, Citizen)

"My overall impressions were very POSITIVE. I like the fact that this is like the encyclopedia for environmental health.” (Shyam, Citizen)

"This is very comprehensive and specific to particular concerns. It's very plain and surprisingly easy to use.” (Wendy, Citizen)

"This is very comprehensive and specific to particular concerns. It's very plain and surprisingly easy to use.” (Wendy, Citizen)

19



Strengths and Weaknesses

Some of the most positive aspects of the Environmental Health and Toxicology site were its clean layout and ease of use.

Respondents also felt that it was a good "starting point" for encyclopedic information on chemicals and toxicology and found it to be very useful.

"I think there are so many uses for this site. I will certainly tap into it for any personal concerns I have about toxins and I think it will be a great education

resource, too.” (Juliana, Citizen)

"I think there are so many uses for this site. I will certainly tap into it for any personal concerns I have about toxins and I think it will be a great education

resource, too.” (Juliana, Citizen)

“Info on specific chemical hazards. Household was good.” (Linda, Citizen) “Info on specific chemical hazards. Household was good.” (Linda, Citizen)

"Overall, homepage pulls together divergent but related topics and displays them in a clean layout. Provides quick access to literature citations and 'ready-

reference' data.” (Heather, Professional)

"Overall, homepage pulls together divergent but related topics and displays them in a clean layout. Provides quick access to literature citations and 'ready-

reference' data.” (Heather, Professional)

"I had no dislike when I visited it.” (Shyam, Citizen) "I had no dislike when I visited it.” (Shyam, Citizen)

20



Strengths and Weaknesses (cont'd.)

Professional respondents encountered an obstacle of missing some of the information below the fold of the screen (requiring scrolling), feeling that the information was buried in layers accessible only after multiple clicks.

In the Citizens group, some complained of inaccessible terminology, FAQs that were insufficient for their needs, and an overwhelming amount of information.

"The top section of the homepage emphasizes NLM resources. It takes several clicks to get to non-NLM resources.” (Nancy, Professional)

"The top section of the homepage emphasizes NLM resources. It takes several clicks to get to non-NLM resources.” (Nancy, Professional)

"I was not crazy about the FAQs-no one ever seems to answer my questions! … I always feel like the questions I have are not on the list and I have to email them to

someone-by that time, I have found it on my own.” (Juliana, Citizen)

"I was not crazy about the FAQs-no one ever seems to answer my questions! … I always feel like the questions I have are not on the list and I have to email them to

someone-by that time, I have found it on my own.” (Juliana, Citizen)

"I wished the primary info that came up was less compound bound inasmuch as I'm a layperson.” (Nancy, Citizen)

"I wished the primary info that came up was less compound bound inasmuch as I'm a layperson.” (Nancy, Citizen)

"A lot is ‘below the fold’ on the screen when displayed with medium sized font. Layered headers at top take up a lot of real estate.” (Heather, Professional)

"A lot is ‘below the fold’ on the screen when displayed with medium sized font. Layered headers at top take up a lot of real estate.” (Heather, Professional)

"Tremendous resource, but a little overwhelming to be presented with everything all at once.” (Shalom, Citizen)

"Tremendous resource, but a little overwhelming to be presented with everything all at once.” (Shalom, Citizen)

21



Strengths and Weaknesses (cont'd.)

Also in the Citizens group, one respondent had difficulty navigating (describing it as having "too much information").

Also in the minority, one respondent in the Citizens group who visited Tox Town disliked the youthful orientation of the design, questioning for whom it was intended.

“wasn't thrilled with the neighborhood picture - seemed a bit juvenile - are you targeting kids as an audience?” (Lori, Citizen)

“wasn't thrilled with the neighborhood picture - seemed a bit juvenile - are you targeting kids as an audience?” (Lori, Citizen)

"The site was hard to navigate and not very user friendly … There is too much information to filter through. The main page could be a link to different, separate home pages, each

specializing in a topic instead of duplication within multiple categories.” (Mark, Citizen)

"The site was hard to navigate and not very user friendly … There is too much information to filter through. The main page could be a link to different, separate home pages, each

specializing in a topic instead of duplication within multiple categories.” (Mark, Citizen)

“I looked at Tox Town but was turned off by the animation.” (Lorraine, Citizen) “I looked at Tox Town but was turned off by the animation.” (Lorraine, Citizen)

"Tox Town was very confusing. I was looking for a section to put my zip code or city in.” (Robert, Citizen)

"Tox Town was very confusing. I was looking for a section to put my zip code or city in.” (Robert, Citizen)

22



Comparison to Other Resources

The majority of respondents felt that the Environmental Health and Toxicology site was superior to other similar resources on the Internet. In contrast to TOXNET, it is perceived as more comprehensive.

Some of the best things about it included its consolidation of information and the ability to search multiple databases via one portal.

"consolidation of all this info and connections at one homebase to search from and to be able to search all tox-related databases at once.” (SueAnn, Professional)

"consolidation of all this info and connections at one homebase to search from and to be able to search all tox-related databases at once.” (SueAnn, Professional)

"Better. I really like having the multiple entry points -- both topical and audience.” (Megan, Professional)

"Better. I really like having the multiple entry points -- both topical and audience.” (Megan, Professional)

"It is better, and easier for me to use … I like it very well as it is.” (Jim, Citizen) "It is better, and easier for me to use … I like it very well as it is.” (Jim, Citizen)

"It actually contains a significant amount of the sites that I use--so it is a single portal--no extra typing.” (Lawrence, Professional)

"It actually contains a significant amount of the sites that I use--so it is a single portal--no extra typing.” (Lawrence, Professional)

"The EHT site may be more inclusive, covering more audience.” (Christopher, Professional) "The EHT site may be more inclusive, covering more audience.” (Christopher, Professional)

23



Suggestions for Change

Suggestions for change included expanding the FAQ, adding mouseovers to the database names, and adding maps and "entrances" for the non-professional user.

"Once I've gotten a few levels down, I tend to ‘get lost.’ I'd like to see every page have a map of the site.” (Nancy, Professional)

"Once I've gotten a few levels down, I tend to ‘get lost.’ I'd like to see every page have a map of the site.” (Nancy, Professional)

“Any way to have mouseovers on database names to tell what's in each database (on blue toxnet bar on right side)?” (Megan, Professional)

“Any way to have mouseovers on database names to tell what's in each database (on blue toxnet bar on right side)?” (Megan, Professional)

"Move toxnet faq higher on screen. add mouse over description of what is behind items.” (Heather, Professional)

"Move toxnet faq higher on screen. add mouse over description of what is behind items.” (Heather, Professional)

"I agree about the FAQ's and there should be different colours or texts to distinguish the different areas because your eyes don't know where they're moving from and to

seemingly.” (Wendy, Citizen)

"I agree about the FAQ's and there should be different colours or texts to distinguish the different areas because your eyes don't know where they're moving from and to

seemingly.” (Wendy, Citizen)

"Maybe a series of relational or cognitive maps to navigate the site.” (Marcus, Professional)

"Maybe a series of relational or cognitive maps to navigate the site.” (Marcus, Professional)

"I would add an 'Entrance' for lay persons, i.e., non chemists.” (Nancy, Citizen) "I would add an 'Entrance' for lay persons, i.e., non chemists.” (Nancy, Citizen)

24



Suggestions for Change (cont'd.)

Additionally, there were suggestions for reorganizing the information in a way that emphasizes PURPOSE or REASON for the search.

One user also suggested including training tools, such as downloads, and more online tutorials.

“It would be helpful to have a link to health and safety training materials and resources, e.g., powerpoint downloads.” (Ron, Professional)

“It would be helpful to have a link to health and safety training materials and resources, e.g., powerpoint downloads.” (Ron, Professional)

"Something to direct me at a high level *first* -- e.g., are you looking for information about chemicals, are you looking for material for teaching a class -- those are so

different, they shouldn't be mashed together on the same page.” (Shalom, Citizen)

"Something to direct me at a high level *first* -- e.g., are you looking for information about chemicals, are you looking for material for teaching a class -- those are so

different, they shouldn't be mashed together on the same page.” (Shalom, Citizen)

25



Target Audience

Most respondents in the Professionals group felt that the site was designed for a professional audience, while those in the Concerned Citizens group felt that it was geared toward both professionals and the general public (with a stronger orientation toward the savvy professional).

One respondent, however, noted that the site requires a certain amount of technical savvy to use.

"bench scientists, regulatory scientists, at least." (Christopher, Professional) "bench scientists, regulatory scientists, at least." (Christopher, Professional)

“Seems like it's trying to meet the needs of everybody, but assuming pretty sophisticated users / technically savvy.” (Shalom, Citizen)

“Seems like it's trying to meet the needs of everybody, but assuming pretty sophisticated users / technically savvy.” (Shalom, Citizen)

“Practitioners in the field--but ToxTown is more for ‘students.’" (Lawrence, Professional)“Practitioners in the field--but ToxTown is more for ‘students.’" (Lawrence, Professional)

“I think it's good for anyone, but will probably be used more by scientists and health professionals." (Nancy, Professional)

“I think it's good for anyone, but will probably be used more by scientists and health professionals." (Nancy, Professional)

"Professionals and laypersons doing research." (Lorraine, Citizen)"Professionals and laypersons doing research." (Lorraine, Citizen)

“Professionals in the environmental health field. That was my first impression - can be used by anyone after looking at it." (Laura, Citizen)

“Professionals in the environmental health field. That was my first impression - can be used by anyone after looking at it." (Laura, Citizen)

26



The Use of Acronyms and its Impact on Perception of Target Audience

Specific to TOXNET, some respondents in the Concerned Citizens group felt that the site was not designed for the "average joe." This finding emerged from users' lack of familiarity with the acronyms used on the site. While the rollovers made a minor difference, non-professional users prefer "at-a-glance" clarity to make these categories more accessible to them.

In contrast, Environmental Health and Toxicology did seem more suited to both audiences. This discussion, however, yielded a finding that it was not obvious that there were two distinct sites.

"The acronyms throw you out of the search immediately, if you don't know what they stand for. It's not clear within a glance which site one would want to use. Toxnet seems better

suited to professionals in .. chemistry, e.g." (Nancy, Citizen)

"The acronyms throw you out of the search immediately, if you don't know what they stand for. It's not clear within a glance which site one would want to use. Toxnet seems better

suited to professionals in .. chemistry, e.g." (Nancy, Citizen)

“Toxnet is only for free form search; it does not list topics or content. Toxnet does not seem like it is for the average joe … When I look at the list of databases I do not know or recognize the acronyms. Some people just want to know specific info and don't have time to learn along the way." (Laura,

Citizen)

“Toxnet is only for free form search; it does not list topics or content. Toxnet does not seem like it is for the average joe … When I look at the list of databases I do not know or recognize the acronyms. Some people just want to know specific info and don't have time to learn along the way." (Laura,

Citizen)

"It’s actually not clear that it is two sites. I was confused while going from one site to the other...asking...am I on the same site... did I go to another site etc." (Linda, Citizen)

"It’s actually not clear that it is two sites. I was confused while going from one site to the other...asking...am I on the same site... did I go to another site etc." (Linda, Citizen)

"it was confusing knowing what site I was on at different times. Maybe sometimes open a new window, so I can still go back to the main portal and try something else?"

(Shalom, Citizen)

"it was confusing knowing what site I was on at different times. Maybe sometimes open a new window, so I can still go back to the main portal and try something else?"

(Shalom, Citizen)

27



Three Searches

Also unclear to many respondents (mostly in the Professionals group) was the fact that there are THREE searches on the site (SIS, NLM, and TOXNET).

"Not clear to me either, why can't the search go through all of them automatically?" (Christopher, Professional)

"Not clear to me either, why can't the search go through all of them automatically?" (Christopher, Professional)

“I guess it wasn't clear to me---just chose toxnet and then NLM separately." (SueAnn, Professional)“I guess it wasn't clear to me---just chose toxnet and then NLM separately." (SueAnn, Professional)

"SIS search isn't as clear because it doesn't have a blank box in which to type and the other two do. The NLM search is up in the header and I've probably already scrolled it off the page to

see more content. Toxnet is clear - it is the topic of the page, it is set off, it lists databases, it has a search box." (Heather, Professional)

"SIS search isn't as clear because it doesn't have a blank box in which to type and the other two do. The NLM search is up in the header and I've probably already scrolled it off the page to

see more content. Toxnet is clear - it is the topic of the page, it is set off, it lists databases, it has a search box." (Heather, Professional)

"I was not clear on which to use." (Jim, Citizen)"I was not clear on which to use." (Jim, Citizen)

28



"Especially For"

The majority of respondents in both groups felt that the "Topics" and "Especially For" sections were very useful starting points. Also appealing were the category list under "Topics" and the highlighting of NEW categories.

Some suggested adding categories to this area, such as breakdowns in the "Public" section for Parents, Kids, and Students. Other suggestions included "Community Volunteers," "Risk Assessors," and "Regulators."

“might add sections for risk assessors and/or regulators." (SueAnn, Professional) “might add sections for risk assessors and/or regulators." (SueAnn, Professional)

“Yes, I started my searches there practically every time.” (Ron, Professional) “Yes, I started my searches there practically every time.” (Ron, Professional)

“I found a lot of links in the pesticide topic.” (Lorraine, Citizen) “I found a lot of links in the pesticide topic.” (Lorraine, Citizen)

“special populations, interest groups? College students? Are they on there?” (Juliana, Citizen) “special populations, interest groups? College students? Are they on there?” (Juliana, Citizen)

“Especially for is excellent, topics is good. The rest is overkill for the general public seeking a quick answer to a simple question.” (Mark, Citizen)

“Especially for is excellent, topics is good. The rest is overkill for the general public seeking a quick answer to a simple question.” (Mark, Citizen)

29



"More to Explore"

Most respondents also felt that the "More to Explore" section was useful. Some (though few) felt that the list was "too long" yet comprehensive. One user suggested adding a "Hot Topics" section to the list, and another suggested organizing the categories by purpose.

Again, in this area, the acronyms became an issue for some in the Concerned Citizens group, despite the aid of the mouse rollovers, and some in this group found the list to be long.

"Organize by what the *user* is trying to do, rather than where the information resource originally came from." (Shalom, Citizen)

"Organize by what the *user* is trying to do, rather than where the information resource originally came from." (Shalom, Citizen)

“I skipped a lot of more to explore seemed too lengthy and many of the topics unrecognizable as relevant.” (Lorraine, Citizen)

“I skipped a lot of more to explore seemed too lengthy and many of the topics unrecognizable as relevant.” (Lorraine, Citizen)

“Although the acronyms are found on the home page in a hard-to-find manner (the question mark), they should be spelled out on each and every page in case someone links to a page or

is referred.” (Mark, Citizen)

“Although the acronyms are found on the home page in a hard-to-find manner (the question mark), they should be spelled out on each and every page in case someone links to a page or

is referred.” (Mark, Citizen)

“How to organize 'more to explore' … might have to simply rethink the page real estate. all valuable items but list is long. More to explore is a mixed list - some are individual resources

and some are links to links.” (Heather, Professional)

“How to organize 'more to explore' … might have to simply rethink the page real estate. all valuable items but list is long. More to explore is a mixed list - some are individual resources

and some are links to links.” (Heather, Professional)

30



"Reference Tools"

Reference Tools received mixed reactions. It was the least explored area of the home page, but those who explored found the "news" and "A to Z" list particular useful.

Some in the Professionals group had questions.

“The resources and the news-everything really-If any of these topics come up in my classes I am going to link them to this website." (Juliana, Citizen)

“The resources and the news-everything really-If any of these topics come up in my classes I am going to link them to this website." (Juliana, Citizen)

“I liked the A-Z section.” (Robert, Citizen) “I liked the A-Z section.” (Robert, Citizen)

“The reference tools seemed adequate for me.” (Mwenda, Professional)

“The reference tools seemed adequate for me.” (Mwenda, Professional)

"Would it be too much to ask for OEL documentation info?” (Lawrence, Professional)

"Would it be too much to ask for OEL documentation info?” (Lawrence, Professional)

"can one access MSDS info from the reference tools?” (Christopher, Professional)

"can one access MSDS info from the reference tools?” (Christopher, Professional)

31



Online Tutorials and Videos

The "online tutorial" concept was well received among Professionals, and -- while many Concerned Citizens showed an interest in this idea -- some in the Citizens group were skeptical.

"Tutorials could be useful -- maybe Flash examples of using the site to find particular pieces of information." (Shyam, Citizen)

"Tutorials could be useful -- maybe Flash examples of using the site to find particular pieces of information." (Shyam, Citizen)

"Didn't watch the tutorial but applaud it being there. We get … questions about importing data into all of the bibliographic management softwares … Could endnote and procite and refworks be added?" (Heather, Professional)

"Didn't watch the tutorial but applaud it being there. We get … questions about importing data into all of the bibliographic management softwares … Could endnote and procite and refworks be added?" (Heather, Professional)

"Would be useful for new PhD students." (Michael, Professional)"Would be useful for new PhD students." (Michael, Professional)

"Would they provide value. Who would use this? Would the user pay to download the video? Who would product it, a professor looking to enhance his career on this area of

interest, the government, or a 3rd party?" (Mark, Citizen)

"Would they provide value. Who would use this? Would the user pay to download the video? Who would product it, a professor looking to enhance his career on this area of

interest, the government, or a 3rd party?" (Mark, Citizen)

"That would be alright-but videos use up a lot of memory, power, etc-especially if you have an older computer or dial-up." (Juliana, Citizen)

"That would be alright-but videos use up a lot of memory, power, etc-especially if you have an older computer or dial-up." (Juliana, Citizen)

"no but am interested in this because we recently have an institutional ‘copy’ of reference manager available; previously had to teach self a pkg Endnote which was time consuming" (SueAnn, Professional)

"no but am interested in this because we recently have an institutional ‘copy’ of reference manager available; previously had to teach self a pkg Endnote which was time consuming" (SueAnn, Professional)

32



Online Tutorials and Videos (cont'd.)

Several respondents said they would be unlikely to watch a video, however. For some, there was an issue with the player required to view the video.

"I would not watch a video, just play around and enter search terms to find my info fast." (Lorraine, Citizen)

"I would not watch a video, just play around and enter search terms to find my info fast." (Lorraine, Citizen)

"perfect...I like the online tutorials, videos sometimes slow the system down." (Robert, Citizen)

"perfect...I like the online tutorials, videos sometimes slow the system down." (Robert, Citizen)

"I didn't try it because my employer hasn't purchased reference manager software ..." (Nancy, Professional)

"I didn't try it because my employer hasn't purchased reference manager software ..." (Nancy, Professional)

33



Trust and Reliability

In the Citizens group only, the issue of trust emerged among a minority of respondents, based on the fact that Environmental Health and Toxicology is a ".gov" site.

This view, however, was balanced by the majority of respondents in this group who DID trust the data.

"I'm sure it's tracked..." (Wendy, Citizen)"I'm sure it's tracked..." (Wendy, Citizen)

"yes for the most part but then most of it was govt sites and I don't completely trust the govt." (Lorraine, Citizen)

"yes for the most part but then most of it was govt sites and I don't completely trust the govt." (Lorraine, Citizen)

"I don't trust gov't sites at all...they have an agenda obviously." (Linda, Citizen)"I don't trust gov't sites at all...they have an agenda obviously." (Linda, Citizen)

"I would hope it would be. It seems like a reliable source" (Robert, Citizen)

"I would hope it would be. It seems like a reliable source" (Robert, Citizen)

"Yes. I see it as a trustable source of info. And reliable." (Shyam, Citizen)"Yes. I see it as a trustable source of info. And reliable." (Shyam, Citizen)

34



Anticipated Usage

With extremely few exceptions, an overwhelming majority of respondents expect to visit the Environmental Health and Toxicology site frequently and, in some cases, have already referred others to it.

"Absolutely-I plan to do an in-service to update my staff." (Jon, Professional) "Absolutely-I plan to do an in-service to update my staff." (Jon, Professional)

"Yes I would. I would make it a point to talk about it in our community meeting." (Shyam, Citizen)

"Yes I would. I would make it a point to talk about it in our community meeting." (Shyam, Citizen)

“Definitely. I have already referred my mother." (Linda, Citizen) “Definitely. I have already referred my mother." (Linda, Citizen)

"I do recommend to my students as an easy-to-access source." (Michael, Professional)"I do recommend to my students as an easy-to-access source." (Michael, Professional)

35



Receiving Updates about Environmental Health and Toxicology

Most respondents in both groups prefer to learn about NLM resources via e-mail.

"Email works fine. Also had a good experience at the NLM booth at a national conference." (Ron, Professional)

"Email works fine. Also had a good experience at the NLM booth at a national conference." (Ron, Professional)

"email updates are great … Perhaps at international scientific/medical meetings. Great stand at Eurotox last year." (Michael, Professional)

"email updates are great … Perhaps at international scientific/medical meetings. Great stand at Eurotox last year." (Michael, Professional)

"Had great exposure at APHA -- wish there had been a formal exposure here at EPA rather than we all discovering bit by bit, but that was what it was like 15-20 yrs ago." (Marcus, Professional)

"Had great exposure at APHA -- wish there had been a formal exposure here at EPA rather than we all discovering bit by bit, but that was what it was like 15-20 yrs ago." (Marcus, Professional)

"Monthly email updates." (Robert, Citizen)"Monthly email updates." (Robert, Citizen)

"Thru' environment magazines. Thru' web engines." (Shyam, Citizen)"Thru' environment magazines. Thru' web engines." (Shyam, Citizen)

36



Receiving Updates about Environmental Health and Toxicology (cont'd.)

Some of the organizations mentioned as appropriate places to get the word out about Environmental Health and Toxicology included the following:

AAPCC

AACT

ALA

CDC

City/county governments

Home Owners Association

Hospitals

MLA

USAIN

Society of Toxicology

Association of Government Toxicologists

Professional organization for agriculture

Following is a series of recommendations that emerged from the results of these online focus groups.

37


I. Background





VI. Recommendations

VII. Appendix

38


Recommendations

A few recommendations that emerged from these focus groups included the following:

Use mouse rollovers as well as on-screen definitions to explain acronyms used on the site.

Clarify the presence of the three search mechanisms (i.e., NLM, SIS, TOXNET).

Break "Public" categories under "Especially For" into sub-categories.

Clarify the points at which users leave one site and go to another. Though design consistency was a clear strength of the NLM sites, many users were unclear as to "where" they were.

In the same vein, providing site maps and other navigational tools (e.g., "bread crumbs") may significantly enhance the user experience.

In communicating the word about Environmental Health and Toxicology, canvas a broad range of both professional and non-professional venues. Most prefer receiving e-mail updates on new additions to the site.

Following are respondent profiles of the participants in these online focus groups.

39


I. Background





VI. Recommendations

VII. Appendix: Respondent Profile

40


Appendix: Respondent Profiles

41


Appendix: Respondent Profiles - Group 1 (Professionals)

# Name Gender City, State Occupation

1 Christopher

M Research Triangle Park, NC

Research biologist

2 Heather F Stillwater, OK Veterinary medicine librarian

3 Jon M Houston, TX Clinical toxicologist

4 Lawrence M Poquoson, VA Medical toxicologist

5 Marcus M Kansas City, MO Environmental engineer

6 Megan F Durham, NC Librarian

7 Michael M Brisbane, Australia Toxicologist, University of Queensland

8 Mwenda F Kansas City, MO Medical doctor

9 Nancy F Saline, MI Chemist/Toxicologist

10 Ron M Denver, CO Industrial hygienist

11 SueAnn F College Park, MD Toxicologist

42


Appendix: Respondent Profiles - Group 2 (Concerned Citizens)

# Name Gender City, State Occupation

1 Jim M Dallas, TX Business consultant

2 Juliana F Littleton, CO Anthropology professor

3 Laura F West Chester, OH Curriculum & instructional specialist for high school math

4 Linda F Salem, OR Collections agent

5 Lori F Queens, NY Administrator

6 Lorraine

F San Francisco, CA Homemaker

7 Mark M Santa Clara, CA Electrical engineering consultant

8 Robert M Norfolk, VA Sales director, Retail Merchants Association

9 Nancy F Portland, OR Retired neuropsychologist

10 Shalom M Washington, DC Energy consultant

11 Shyam M Sioux Falls, SD IT Manager, Science Applications International Corp.

12 Wendy F Milwaukee, WI Homemaker

43


Contact:

Mary Beth Solomon

For inquiries and capabilities on

Qualitative Analytics

201.434.0404917.601.1273

Contact:

Mary Beth Solomon

For inquiries and capabilities on

Qualitative Analytics

201.434.0404917.601.1273

NLM TEHP Final Report Solomon Solutions [email protected] Voice 201.434.0404 Mobile:...

Documents

Transcript of NLM TEHP Final Report Solomon Solutions [email protected] Voice 201.434.0404 Mobile:...