Reliability and validity …And why it is important!

COUN 6360 Dr. Linda Foster

This information will help you learn more about reliability and validity for all the interactive case studies.

Reliability

• According to the Erford, “Reliability means consistency” (2013, p. 129). A reliable assessment instrument produces similar results when taken multiple times by the same test taker. For example, if an assessment instrument is designed to measure intelligence, the results should be approximately the same each time the instrument is administered to a specific individual.

Validity

• Validity refers to whether an assessment instrument measures what it is intended to measure. The intelligence test used in the example above is valid only if the assessment instrument actually measures intelligence. Without reliability and validity, test scores would lack integrity and could lead to faulty results, which may ultimately harm test takers.

Reliability, validity and assessment instruments

• By looking at the reliability and validity numbers you can be assured that an assessment has been thoroughly tested and hopefully will provide you with the best results to help your client. Usually the reliability and validity information is reported in the test manual that accompanies the actual test. In addition, you can find this information on the test publisher’s website as well.

What about intake interviews? • One of the most important aspects when establishing

the counselor/client relationship is beginning with an in-depth and thorough intake interview. The intake interview provides the counselor with detailed information regarding the client’s physical, emotional, spiritual, and psychological background and history. This information provides the counselor with a foundational background, including multicultural considerations essential to creating a treatment plan.

Intake interview

• For example, a first-time client enters the counseling office complaining of the inability to get a decent night’s sleep. During the intake interview, the client acknowledges that she drinks the equivalent of two pots of coffee and two 2-liter containers of caffeinated soda every day. Had the intake interview not provided this information, the counselor may have suggested the client seek medical advice and a sleep aid from her physician to help her sleep. Instead, by the counselor suggesting the client alter her intake of caffeine, the client may have some immediate results not requiring drug intervention.

Mental status exam

• Similarly, if a client comes to a counseling session exhibiting unusual, uncharacteristic, or bizarre behavior, it would be of little or no value to proceed with an intake interview or a regular counseling session without first administering a mental status exam. Mental status exams, performed regularly, can be an effective check-in to establish whether a client is progressing or digressing throughout the counseling relationship.

Beginning assessments

• Both of these assessment interviews are invaluable tools in the practice of counseling. As a future counselor, you must understand the appropriateness of using an intake interview and a mental status exam in the best service of your clients.

Validity of the instrument you are using

• Neukrug and Fawcett (2006) provided a helpful way to consider the validity of informal assessment tools: “The more clearly one is able to identify the kinds of behaviors and the place the behaviors are exhibited, the easier it will be to collect information about the domain and consequently, the more valid will be the assessment” (p. 209).

Face validity

• So, it is important to ask the right question to get the right (valid) information. Does that make sense?

• Face validity – has to do with the superficial “look” of the instrument.

• Does the instrument “appear” to be measuring what it is supposed to measure?

Garbage in and garbage out

• So, one aspect of validity is face validity.• You would not use an intelligence test to

determine if someone is depressed… but instead would use a depression inventory.

• The more clearly defined questions will result in better (valid) information from the client.

So what does this have to do with intake interviews and mental status exams?

• Intake interviews and mental status exams can be thought of as informal assessments.

• And valid instruments – right? Because you are gathering information…

• So, face validity can be used for both of these instruments.

Validity of the instrument used in assessment

• For the other instruments used throughout the term… you will learn about other types of validity.

• Validity is measure and described in different ways: – Content– Construct– Criterion-related

Validity: whether the test measures what it is supposed to measure.

• Validity takes the form in various ways: content validity, construct validity, criterion related validity and even basic face validity.

• You must look at articles, the MMY, or even the test publisher’s website to find out about validity of any instrument you use.

In some cases, validity will be reported in one of those areas – maybe in all of them.

• Validity is the attempt by the test developers to make sure the test measures what it states it is measuring.

• Usually it is reported in the Mental Measurements Yearbook or on the test publisher’s website.

• You want to make sure the instrument you are using is indeed intended for your purpose