CSQ Administering & Scoring 03 05 12

February 1, 2020

Clifford Attkisson, Ph.D.

Administering and Scoring the CSQ Scales®

Inquiries: [email protected]

Web: www.CSQscales.com

Delaware Place # 3205, Chicago, IL 60611-1629 USA. Voice: 415-310-5396. Fax: 339-

440-9537. U.S. Domestic Toll-Free Fax: 866-770-4975.

Available from Copyright Holder. Scale use is contingent on express written permission

from the copyright holder (Attkisson) and remission of use fees. Permission and payment

of fees are ordinarily obtained and made via the website (www.csqscales.com). A formal

license agreement is required for all electronic platform formats and other selected forms

of use of scales or selected scale items. License agreement information and procedures can

also be found at the website.

Cost: For English versions cost per use is $.55 each for first 500 uses, $.45 each use

thereafter. Pricing for orders less than 500 uses and for translated versions in languages

other than English varies and is higher due to costs of research, translation, production,

and printing. Introductory prices may be available for a limited period of time following

launch of our new website in the year 2020. You are advised to check the site regarding

pricing, introduction of additional products and services, and information regarding use

and development of the CSQScales®.

CSQ Scales® Overview

The CSQScales® were created in response to the need for a standard instrument to replace

idiosyncratic, ad hoc, and/or untested tools. The goal was to develop a standardized

measure with strong psychometric properties that could be used to assess general

CSQ Scales Administration & Scoring

Clifford Attkisson, Ph.D.

February 1, 2020

Page 2

satisfaction across varied health and human services. The CSQScales® (CSQ) include a

coordinated series of brief instruments. The CSQ is documented to have excellent

reliability and internal consistency. The CSQ is reported to have high levels of client and

staff acceptability when tested in numerous studies involving diverse client samples and a

wide range of health and human service programs. In summary, the major strengths of the

CSQ include its utility as a standard measure, excellent reliability and internal consistency,

acceptability to clients and service providers, and sensitivity to different levels of program

quality, and value to service providers committed to enhancement of quality and impact of

services (Attkisson & Greenfield, 1996, 2004; Attkisson & Pascoe, 1983; Attkisson &

Zwick, 1982; Greenfield, 1983; Larsen, Attkisson, Hargreaves, & Nguyen, 1979; Nguyen,

Attkisson, & Stegner, 1983).

Administering CSQScales®

Applicable Populations and Service Types

The measures have been adopted in quality assurance, evaluation research, and services

research studies across a wide range of health and human service programs. Service

settings studied include outpatient and inpatient mental health facilities, public health

center clinics, primary care health clinics, eating disorders clinics, cancer and cardiac

research services and research, health maintenance organizations, employee assistance

programs, police and criminal justice services, legal services, mandatory short term alcohol

abuse treatment programs, residential alcoholism treatment programs, community-based

residential care, case management for the individuals with severe mental disorder, and with

AIDS self-support and psycho-educational groups (Attkisson & Greenfield, 1996, 2004;

Greenfield, 1983; Pascoe, 1983)

Applicable Age Groups

• Direct ratings are elicited from adolescents and adults.

• Parents and caretakers are often respondents about services provided to

children.

• A parent-rated version of the CSQ-8 is available for use.

• A child version, using standard wording and expressive faces, is available

for use.

CSQ Administration (CSQ-3, CSQ-4, CSQ-8, CSA-18A & -18B)

The CSQ Scales® are self-administered, with data collected usually at the end of services

or at desirable temporal intervals during service delivery. Items are questions inquiring

about respondents’ opinions and conclusions about services they have received or are

currently receiving. Response options differ from item to item, but all are based on a four-

point scale. Examples* include “How satisfied are you with the amount of help you have

received?” (for which the response options are 1=”Quite dissatisfied”, 2=”Indifferent or

mildly dissatisfied”, 3=”Mostly satisfied”, 4=”Very satisfied”, and “Have the services you

received helped you to deal more effectively with your problems?” (Which has the

CSQ Scales Administration & Scoring

Clifford Attkisson, Ph.D.

February 1, 2020

Page 3

responses 4=”Yes, they helped a great deal”, 3=”Yes, they helped somewhat”, 2=”No, they

didn't help”, 1= “No, they seemed to make things worse”. All items are positively worded;

however, the directionality of response options span the spectrum from very negative to

very positive; and, the numerical anchors for items are reversed randomly (from high to

low satisfaction or low to high satisfaction within each item) to minimize stereotypic

response sets. While addressing several elements that contribute to service satisfaction, the

CSQ-8 has no subscales and yields a single score measuring a single dimension of overall

satisfaction (Larsen et al., 1979).

*Item content reprinted with permission of copyright holder.

Administration Time

Reported tests –3 to 8 minutes; author tests–1.5 minutes.

Scoring CSQ Scales® (CSQ-3, CSQ-4, CSQ-8, CSA-18 A & B)

Scoring CSQ Scales®

An overall score is calculated by summing the respondent’s rating (item rating) score for

each scale item. For the CSQ-8 version, scores therefore range from 8 to 32, with higher

values indicating higher satisfaction. Scoring for other versions is similar after

extrapolating for number of items.

Scoring the closed-ended part of the CSQ Scales involves: (a) unweighted

summation of the direction-corrected response values (1–4 for all the CSQ scales) for the

total scales; and (b) calculation of measures of central tendency (such as mean, standard

deviation, median, and mode) of the individual item ratings and for the total scale scores.

Scoring is not complicated and involves calculation of total score across all items for each

subject and analyzing item and total score distributions across groups of subjects. Because

of the scale's single factor structure, interpretation of CSQ data involves a straightforward

comparison of results obtained for a given service or client group with external data that

constitute an appropriate frame of reference, e.g., the multi-service setting means and

standard deviation results presented in Nguyen, Attkisson, & Stegner (1983) or Attkisson

& Greenfield (2004).

Administering CSQ Scales®: General Considerations

Data gained from the CSQ Scales® are typically self-completed but aural responses have

been collected from individuals with serious disorders in hospital acute care, day treatment,

and case management studies (LeVois, Nguyen, & Attkisson, 1981).

Methodology and administrative procedures for using the CSQ Scales® are

relatively straightforward. Mail survey methods (either mail-out and prepaid mail-back, or

hand-out and mail-back) have sometimes been used for collecting the data. The main

disadvantage of these approaches is low reported response rates, 40-50% being the highest

typically achieved in instances with one follow-up postcard reminder. The recommended

approach is to use point of service or waiting room surveys with a designated scale

administrator or a receptionist trained in procedures for systematically soliciting voluntary

participation from sampled clients. Various sampling protocols have been used: systematic

CSQ Scales Administration & Scoring

Clifford Attkisson, Ph.D.

February 1, 2020

Page 4

or random samples of client rosters, samples stratified by duration of services so far

received, and census samples of all clients seen during a specific time frame (Attkisson &

Greenfield, 1996, 2004). The census sampling approach, provided the time interval is at

least two typical service weeks, has the advantage of assuring few clients are omitted (only

those missing appointments throughout the whole period, or those declining to participate).

Completion rates tend to be above 90% when this more satisfactory approach is used,

reducing the risk of unknown non-resource bias. Conversely, such face-to-face methods

generally include clients whose treatment is in progress.

The practical problem with surveying satisfaction at increasing time intervals after

termination or completion of a service program is the expected attenuation in response rate

— a problem likely to confound interpretation of results. Most studies have used

anonymous methods though some have not, and have included code numbers allowing

linkage to service data. One methodological study with the CSQ found optionally identified

(name written in at the option of the recipient) forms did not result in lower response rate

or higher reported satisfaction (Greenfield, 1983). Despite the range of alternative

approaches, the standard waiting room method meets well the simplicity and uniformity of

implementation criterion.

Using sampling and time-series methods, satisfaction levels can be compared

across different service modalities, duration of service, types of clients, providers, or

specific facilities. The CSQ is used in all levels of primary care, mental health care, and

many other human service settings. In using any consumer satisfaction measure, perhaps

the most important validity consideration is designing procedures to obtain high response

rates to minimize biases attributable to non-response.

Reading Level

The reading level of the CSQ Scales® has consistently been found to be at a level that is

accessible to individuals reading at the 5

grade level or higher. Several reported results

are presented below:

Flesch-Kincaid grade level is 4.7 (reported by independent evaluator)

Flesch-Kincaid grade level found by publisher is 5.3

Flesch Reading Ease Index is 69.4

Comparison Data

Means, modes, medians, and standard deviations are available from a series of studies

involving approximately 8,000 clients (Nguyen et al., 1983). Most of the studies also report

information on the demographics of sample members. The diverse subject populations

enrolled in baseline studies include a broad spectrum of demographic characteristics, a

wide range of service types, and variability in amount of services received (Attkisson &

Greenfield, 1996, 2004; Attkisson & Zwick, 1982). The CSQScales® Reprint Portfolio

contains reports and reprints from a wide range of studies that can guide establishment of

appropriate comparison data. The portfolio can be purchased on the CSQScales® web site

(www.csqscales.com).

CSQ Scales Administration & Scoring

Clifford Attkisson, Ph.D.

February 1, 2020

Page 5

Scoring the Service Satisfaction Scale (SSS)

(content available via email: [email protected])

Managing Missing Data in Service Satisfaction Data Sets

This coverage of the “missing data” topic will not be technical as it must be necessarily

general in nature pending more specific information about the design of your current study

and one cannot presume to make more technical suggestions without that knowledge. First,

and foremost, in the future, as you move to new projects, you will want to establish a set

of “a priori” rules about how you will manage missing data and ambiguous responses. This

will be important so that your study methodology will not be biased by your choice of

methods for handling one of the most frequently occurring challenges to data analysis

(regardless of measurement tool or method of administration). More technical approaches

are known and can be conveyed to you, upon request for consultation, in advance of

planning future investigations — especially those involving larger numbers of patients

(subjects) and control conditions or comparison analyses. Tamalpais Matrix Systems, LLC

can link you with an expert in this field, one who has analyzed CSQ data extensively, who

can provide additional consultation (Bruce Stegner, PhD). For now, for a current or

completed project, you may want to consider the following:

(a) Include patients who answer 4 or more items on the CSQ-8 (or a similar ratio

on more lengthy versions). You will need to enumerate and report the number of subjects

meeting this criterion and estimate the effect on overall results that are reported;

(b) For patients meeting criteria # 1, for the missing items assign the average score

for the items that are scored. (This procedure is questioned by some but will allow you to

proceed with your initial project and include these patients. If there are only a few such

patients (subjects), the impact on the results may not be profound and can be measured).

satisfied score checked. Remember, the CSQScales® typically yield negatively skewed

distributions of scores (not a normal distribution but one where responses tend to cluster at

the positive end of the scale — hence, a negatively skewed distribution). So, by choosing

the least satisfied response, when multiple responses are selected by the patient (or subject),

the scores in general are more normalized. You could, alternatively: assign the mid-point

between the two responses selected, e.g., if responses “1” and “2” are selected, assign a

score of “1.5”. This, however, biases the overall results slightly to the positive end of the

score range.

You can see why “a priori” rules are important.

You may also want to consult a statistician who can advise you about alternative or more

CSQ Scales Administration & Scoring

Clifford Attkisson, Ph.D.

February 1, 2020

Page 6

technical approaches. As noted in the first paragraph above, Tamalpais Matrix Systems,

LLC can also refer you to additional resources for consultation on data collection,

preparation of data for analysis, and statistical analysis. Tamalpais Matrix Systems, LLC

hopes that you find this response to be helpful in your work. The response is provided as a

courtesy, cannot be warrantied as the most optimal approach to your specific challenge(s),

and with the understanding that you may want to seek additional consultation from TMS

or others.

Transforming CSQ-8 Data: Displaying Scores as a Distribution of

25 to 100

It is possible to use a variety of data transform methods to enhance the presentation and

understanding of CSQ data. One linear procedure is to compute the total score, for

example with the CSQ-8, by adding up the individual scores from the 8 items and the

multiplying by 3.125 to obtain a distribution from 25 to 100. (A similar approach can be

used with the other CSQ or SSS versions.) I suggested this linear transform of the raw

CSQ-8 scores to a colleague as a mechanism for displaying the scores in the generally

familiar zero to one hundred "school room" format. I further suggested that my colleague

might consider converting the transformed scores into percentiles and then treating each

quartile as a level of relative satisfaction. If you implement this approach, please let me

know how this works out. With repeated use of the CSQScales® over time you can use

your own setting as its own control. Then recruit a sister setting to do the same and make

comparisons. Please keep in touch as you proceed. You may find that your score

distribution is negatively skewed with the proportion of satisfied clients being greater

than the less satisfied clients. In this case, the percentile quartiles will assist in

segmenting levels of relative satisfaction and dissatisfaction.

Following a time series methodology (repeated administration of the CSQ Scales®

over time), service agencies can study trends in service satisfaction results internally or in

comparison with CSQ data from peer settings. CSQ score distributions may be negatively

skewed (with the proportion of satisfied clients being greater than the proportion of

satisfied clients). In this case, the percentile analysis will assist in segmenting levels of

relative satisfaction and dissatisfaction. This approach will work best with large numbers

of respondents and where comparisons can be made between comparable service settings.

Consult your local statistician for additional ideas about linear and non-linear data tranform

possibilities.

Key References

Larsen, D. L., Attkisson, C. C., Hargreaves, W. A., & Nguyen, T. D. (1979).

Assessment of client/patient satisfaction: Development of a general scale. Evaluation and

Program Planning, 2, 197-207.

Attkisson, C. C., & Zwick, R. (1982). The Client Satisfaction Questionnaire:

Psychometric properties and correlations with service utilization and psychotherapy

outcome. Evaluation and Program Planning, 6, 299-314.

CSQ Scales Administration & Scoring

Clifford Attkisson, Ph.D.

February 1, 2020

Page 7

Attkisson, C.C., & Greenfield, T.K. (2004). The UCSF Client Satisfaction Scales:

I. The Client Satisfaction Questionnaire-8. In M. Maruish (Ed.), The use of psychological

testing for treatment planning and outcome assessment (3

. Ed.). Mahwah, NJ: Lawrence

Erlbaum Associates.

Attkisson, C.C., & Greenfield, T.K. (1996). The Client Satisfaction Questionnaire

(CSQ) Scales and the Service Satisfaction Scale-30 (SSS-30). In L.I. Sederer & B. Dickey

(Eds.). Outcome assessment in clinical practice. Baltimore: Williams & Wilkins.

Greenfield, T.K. (1983). The role of client satisfaction in evaluating university

counseling services. Evaluation and Program Planning, 6, 315-327.

LeVois, M., Nguyen, T.D., & Attkisson, C.C. (1981). Artifact in client

satisfaction assessment: Experience in community mental health settings. Evaluation and

Program Planning, 4, 139-150.

Nguyen, T.D., Attkisson, C.C., & Stegner, B.L. (1983). Assessment of patient

satisfaction: Development and refinement of a Service Evaluation Questionnaire.

Evaluation and Program Planning, 6, 299-313.

Clifford Attkisson, Ph.D.