OCCC Safety Perception Worksheet

Program Development
Safety
r
p
0
Surveys
What to Ask, How to Analyze
By Michael O’Toole and David P. Nalbone
T
;?””.–=’-.—-~
1-‘
-Employee safety perception surveys measure employee perceptions about
key factors or domains of
an organization’s safety
management system.
••Such surveys provide
additional vital information
that identifies areas which
need attention and enables
management to improve
an organization’s safety
process.
items
lation.
he fundamental
management
process is to allocate available resources
.
to a productive end. In the case of
s~fety al~d health, management mu:t id~ntIt: how to best allocate limited resources
to. ensure the tewest mishaps that result 111
lllJunes to employees, damage to eqUlpment or harm to the el:Vlronment.
Research suggests that the satetv managenlent system has the most 51gnlhcal~t
Impact on injury rates (Carder, 200.);
O’Toole, 2002). Other research involving
safety management svsterns suggests that
the most critical factor int1uencing suecesstul satetv results IS that ot managemerit’s demonstrated
support of safety
. (Erickson, 2008). Based on her earlier research, Erickson (1994) also suggests that
interventions targeting only safety-related
are less than successful if addressed in isoIn other words, problems or issues identi-
;;-~~.~~~~’~~~
I Michael O’Toole, Ph.D., is an associate professor of safetyscienceat Emory-Riddle
jJ Aeronautical Universitywhere he teaches courses in occupationalsafetyand health.
~JA professionalmember of ASSE’sCentral FloridaChapter, he is a former administra-
tor of the Society’sAcademicand Mining practicespecialties.O’Toole holds a Ph.D.
in PublicHealth from the Universityof Illinois,Chicago,and an M.S. in SafetyEngineering from Northern IllinoisUniversity,as well as an M.A.in Industrial Psychology
and a B.S.in BehavioralPsychology,both fromWestern MichiganUniversity.
David P. Nalbone, Ph.D., is an associate professor of social psychology at Purdue
University, Calumet. He earned his undergraduate degree from the University
at Buffalo,and holds a master’s and Ph.D. in SocialPsychology from Claremont
Graduate University. His teaching interests include introductory and social
psychology, research methods and statistics, and stereotyping and prejudice. His
research interests include terror management theory applications to political and
social issues, and assessment
58 ProfessionalSafetyJUNE
issues in education and organizations.
2011
www.asse.org
fied as safety related are really a symptom of
a broader management svstern issue, such as
leadership and/or visible support for safety
and health issues.
Zohar (1980; 2005) used employee safety
questionnaires
to identify the relative importance of specific safety factors in several industrial settings in Israel. Bailey and Petersen
(1989) used the Minnesota Safety Perception
survey to identify factors that positively contribute to injury reduction in the railroad industry as
well as in several other industries. Results of Bailey’s (1997) follow-up study suggest that at facilities with low injury rates, employees’ perceptions
of critical safety factors were highly positive.
Perceptions,
like attitudes, have been recognized as an important factor in safety. Research in
this area suggests that when measured, perceptions can predict the likelihood of certain behaviors (Ivers, Senserrick. Boufous, et al.. 2009). The
importance of this factor is especially critical where
employees have little or no direct supervision.
In such settings, an employee makes important
choices and decisions about safety rules, practices
and procedures. If perceptions about safety are
low, that employee may be more likely to take a
shortcut or engage in some other at-risk behavior
which can lead to an injury.
Other research suggests that employees’ safetyrelated perceptions are predictive of organizations’
safety results (Carder, 2003; O’Toole & Nalbone.
2007; Sea, 200:5). ‘Nhere employee perceptions of
an organization’s safety climate are low (negative).
the incidence of injurv tends to be higher than in
those organizations
where employee safety perceptions are high (positive).
These results clearly are supported
bv Bailev
(1997) and Bailey and Petersen (1989). Currentlv,
most organizations use some form of a trailing indicator, such as injury incident rates, to measure
the success or failure of safety processes and programs. Some consider use of a safety perception
survey as a leading or predictive indicator of the
success or failure of safety processes and programs
(Carder, 2003). Others view such a survey as a
tool to help an organization continuously improve
SH&E efforts (O’Toole, 2002).
Determining What Type of Survey to Conduct
Once an organization decides to deliver an employee safety perception survey, it faces several decisions. The first is whether to use an off-the-shelf
instrument or develop the survey locally. Each approach has inherent strengths and weaknesses.
Off-the-Shelf Surveys
Off-the-shelf surveys should have undergone a
formal development process that should provide
several important details.
Reliability
A properly developed and tested survey instrument provides users with a high degree of confidence in the information collected via the survey.
Reliability, or the assurance of repeatable results
(also known as precision) with the use
of the same instrument in the
future, is vital.
Brieflv, reliability .refers to
consistency of
responses,
in
terms of both
the people providing those responses and over
time.
If a survey does
not produce repeatable/stable results over
time, its use becomes
suspect. Ideally, the organization will use some or all
of the survey results to improve
the existing safety process. If the
survey instrument is not reliable,
valuable resources may be wasted on
activities with less-than-desired
outcomes.
Several factors affect the reliability of survey
results, and numerous guides have been written
on how to conduct surveys (e.g., Fowler, 1995). For
example, unclear questions or questions for which
there is not a wide range of responses (e.g.. when
there is a socially desirable, or “good” answer)
hamper reliability. Thus, using a broad range of
questions-as opposed to a 1110renarrow range, or
just a few questions-tends
to improve reliability.
www.asse.org
JUNE 2011
ProiessionalSaiety 59
If reliability is
lacking, validity
also will likely be
lacking. Therefore,
producing reliable
I
!
I
l
Figure 1
duce additional bias that adversely affects
the power of the results.
Accuracy vs. Precision
In-House
Surveys
A company also may choose to develop
Inaccurate
Accurate
a home-grown employee perception surresults is a neces(systematic error)
sary (but not sufvey. This decision presents several chalPrecise
ficient) condition for
lenges. For example, the company must
establishing validity.
determine what constructs or factors the
survey will measure. Although the survey
is intended to measure employee perceptions of safety, this broad construct is better measured by identifying several more
specific constructs, such as employee perImprecise
ception of management’s commitment to
(reproducibility error)
the safety process; employee perceptions
of their coworkers’ commitment to the
safety process; employee perceptions of
the effectiveness of safety-related training; and employee perceptions of their
involvement in the organization’s safety
processes.
Conducting a pretest before full-scale impleAnother issue is the timing and frequency of
mentation of the survey can help gauge whether
survey use. After administering the initial survey,
the questions are clear and whether the intended
having employees complete the survey at regumeaning of the survey and its questions are being
lar
intervals (typically every 18 to 24 months) will
understood by those intended to take the survey.
provide management an additional metric against
This useful step helps reduce the likelihood of diswhich to measure SH&E processes. This type of
covering reliability problems later on.
measure will typically give management a leading
indicator to use with the traditional lagging indicaValidity
If reliability is lacking, validity also will likely be tors (Blair & O’Toole, 2010).
lacking. Therefore, producing reliable results is a
Question Development
necessary (but not sufficient) condition for estabOnce constructs are identified, the organizalishing validity.
To ensure that the survev instrument is valid, tion must develop questions that measure those
constructs in an appropriate way. The wording of
one must determine that it accurately measures
questions must elicit a meaningful response withwhat it is intended to measure. If the survey is not
out suggesting or guiding the participant to a “corvalidated, then the results may not be useful and
likely will result in resources being expended on rect” answer, while still allowing for a range of
‘.~*@”‘ ••••gaps or weaknesses that mayor may responses to ensure enough variability among re. S
I
not be reflective of the actual state of sponses to be able to detect any significant effects.
The organization also needs personnel with some
, amp.e
.
the organization’s safety processes.
statistical knowledge to ensure that the survey is
~ Questions
Establishing reliability and validity
measuring the intended constructs in a way likely
of a survey instrument is time (onOSome of the safety and.
suming and resource intensive. ‘!Then to generate meaningful and useful results.
A common problem among first-time survey dehealth procedures/instrucusing an off-the-shelf survey, users
tions do not need to be
pay for the value of knowing that they velopers is that they develop a set of questions, but
do not simultaneously keep in mind the analytical
followed to get the job
are purchasing a valid, reliable instrustrategy that will be required to make use of the
done safely.
ment. However, this advantage must
data, or of how the results of the analysis will assist
OAccident investigations
be weighed against the potential diskey decision makers in determining what changes
are mainly used to identify
advantage of not being able to cus(if any) are needed to the safety programs or prowho to blame.
tomize the survey to specific interests.
cesses. Failure to keep such issues in mind can lead
Changes
OManagement only bothto wasted time and effort on an instrument with
Often, a company may wish to little redeeming value.
ers to look at safety and
change the wording of a question, or
health after there has been
Advantages of a survey developed in-house are
to add or eliminate questions. Such
an accident.
that it requires less upfront financial investrnent
changes can be made, but usually at and the company can customize the level and tone
OThere is nothing I can do
a cost. Additionally, depending on of questions to match the target audience. As with
to further improve safety
the nature of the changes, factors an off-the-shelf survey, validity and reliability are
and health here.
(those things the company is trying
key concerns which n~ust be addressed bv s~meto measure) within the survey could
one with a strong statistical background to ensure
OSafety and health meetbe altered related to their validity and
that the results provide meaningful information
ings are a waste of time.
reliability; the changes also may {ntro- and do not waste resources.
60 ProfessionalSafep{JUNE 2011 www.asse.org
~,.
r;
Developing a high-quality
survey is often an iterative process and may require a longer
time horizon and incremental
cost than most SH&E professionals and managers are willing to commit to this process.
Figure 2
Likert Scale f;)(Cll’llpl,e
Sample Size
Regardless of the type of
survey used, sample size is a
consideration. In smaller facilities, having the entire workforce complete the perception
survey may be appropriate; in larger facilities, a
sampling procedure may be developed to ensure
adequate representation of all important groups
that are being surveyed. A large organization could
elect to have all employees participate in the survey. A critical factor to survey success is to ensure
that enough responses are collected to provide sufficient statistical power to detect any differences
(either over time or as a result of an intervention).
Using the Results
Once the survey has been administered, the organization must decide how it will use the results
to improve current safety processes and programs.
One key to the use of such a perception is to not focus too hard on the degree of positive perceptions,
or the gaps alTlong or between various employee
groups. Rather, management must focus on how
employees developed the perceptions of concern.
Based on the authors’ experience, management
teams often become preoccupied with why all employees or a specific subgroup may have a particularly low perception of a given measured factor.
They then set on a course to convince employees
that their perceptions, at least of safety, are wrong.
Perceptions are similar to attitudes and are difficult to change when attacked head-on. Thus, the
data analysis should direct efforts toward identifying how employees ll1ay have a less-than-positive
perception of a given factor. After all, the perception may be accurate even if it is difficult for management to hear. From there, it may be possible
to create a remedy to improve the perception of a
low-scoring factor.
Another concern is a fear that the survey will
reveal poor practices of supervisors or managers.
Although issues of management style or approach
may drive a particular set of perceptions, that
knowledge may offer senior managers the opportunity to provide resources in the form of training,
education and mentoring to alter or improve the
behaviors or practices of concern.
As with most survey research, confidentiality is
important and must be addressed. Since information requested also may imply either poor employee
practices or behaviors which violate safety rules,
care must be taken to ensure that participants’ responses will not result in termination or other sanctions; otherwise, workers may provide inaccurate
responses based on fear of possible repercussions.
Ensuring confidentiality often begins with a
well-crafted memo or cover letter outlining the
value of employees’ input. Participants must understand that to improve safety procedures or processes, honest responses (as opposed to those that
sound like the “right” answers) are needed. Strong
assurances of confidentiality of all responses will
go a long way to that end.
A key aspect of the use of perception surveys is to
provide feedback to managers as well as employees. If employees do not receive a general summary
of the results or information about how the results
will be used, they may be less inclined to complete
such a survey in the future, and also may be suspicious of its purpose.
Good feedback provides employees with a general, nontechnical explanation of the findings, a
summary of any changes to be implemented as a
result of the findings, and a chance to eX’}Jressany
concerns or questions about the survey process or
the results. Providing employees a chance to have
their views heard (and retlected back to them)
should help improve employee buy-in to the safety
perception survey process.
Likert-type scales,
using anchors on
the ends (e.g.,
strongly disagree
to strongly agree)
are often used;
middle scores can
be labeled (e.g.,
mildly agree) or
not, provided there
are 5 to 7 response
choices and a number corresponding
to each.
Analysis & Evaluation
Determining what types of analyses to run is a
key element of survey development and planning.
To effectively use the information gleaned from the
survey, it is “vise to consider what processes or procedures might be informed by the survey results.
Typically, a handful of questions are written that
address a given area; these are then averaged and
used as a benchmark for that particular area, with
future data compared to that benchmark. Using
5 to 10 benchmark scores should keep the survey
process manageable, and allow supervisors front
different areas to highlight the parts of the survey
analysis that are particularly relevant to their areas.
A final element of the survey cycle is reevaluating the entire process once complete. What was
learned and what remains unknown? What effect
did changes made based on information gained
from the initial survey have on measure able outcomes? What improvements can be made to the
process to make it more effective or efficient?
Thoroughly documenting the results of the reevaluation can be useful, especially if significant
time has elapsed between the end of one survey
cycle and the start of the next. A perception survey
www.asse.org
JUNE 2011
ProfessionalSafety
61
is a tool intended to drive continuous improvement
and should not be considered a one-time fix.
Conclusion
The SH&E profession and government
regu-
r”g.~:=’!IR-e¥II!iOi1I;:~wb~’~~I~Il&1~=&-;”alidity
of Safety Surveys
-Any
type of measurement, no matter its content or format, must
be’ concerned with two psychometric issues: reliability and .
validity.Reliability refers to consistency of scores. If respondents
answer the same (or similar) questions with different responses,
theirscore~areunreliable.
Using a more general example, if one’
steps onabathroom
scale, then off and back on, one should get
the same weight reading (unless one has added C![ removed cloth- “.
ing): if I19t, the- scale-is unreliable.
,:
However, while reli.ability isa necessary condition for demonstrating validity, which is concerned with whether one is measuring what one intends to measure, it.is not enough, Using afypical
bathroom scale, one can get reliable results, but only for weight. A
bathroom scale cannot provide valid information about an individual’s height, intelligence or any other characteristic.
When. developing a survey, both reliability and validity must be
considered before the survey is administered. To address validity,
one could present survey questions to experts within a given area
(e.g., safety inspectors, management) to see whether they agree
that the questions address the desired constructs.
.
Reliability is generally only assessed after data collection, by
internal consistency, to determine whether items on a given scale
. are all correlated with each other (as they should be, if they all tap
into the same construct), or by test-retest or alternate-forms of reliability, to see whether different administrations or forms of a test
produce similar responses (i.e., are correlated).
.
Generally speaking, all item responses must be on a continuous
scale (i.e., multiple choice items will not work well), and should
provide a reasonably large range of possible responses (5 to 7
seems to.work well). Shorter response forms (e,g., “do you agree
.or disagree that. . 7”) do not provide enoughmiddle ground, and
longer forms seem to provide too much.complexity for people to
, locate thE’irno~ponses.
.
‘. ., . ..,.
.:
, ‘Likert~tYre.scales (Figure 2; .p. 61),usiTlg’anchors’on the ends
(e.g.,stropgIy disagree to stronglyagree) are often used; middle
scorescan be labeled (e.g., mildly agree) or not, provided there are
.5 to 7 response choices and a number corresponding to each (e.g.,
“I think that my fellow workers follow safe work practices. Strongly
‘Agree 123..
4 56 Strongly Disagree.”
.
Finally; sampling must be considered. Collecting data from sufficient sample size is important; gathering only a few responses in
a company of thousands will not likely yield useful information on
employees’perceptions
or behavior.
For most basic surveys, a target of 100 or 150 responses should
provide sufficient statistical power to detect any effects that are
present K’ohen, 1992). However, for companies with fewer
employees, the goal should be to sample all employees. In addition, for the sample tobe extemallyvalid (i.e., generalizable), it
must adequately represent the population of workers from which
it is drawn.Thus. sampling only managers will only tell one about .
those-managers, but not about line employees.’
.
As such, recruitment efforts should ensurebroad participation,
perhaps-by inCluding small incentives or by making it clear that the
company values employee feedback.will use it to improve the com’pany; and will keep all responses corifidenti~l to protect employees
and to encourage them to provide honest input.
62 ProfessionalSafety
JUNE 2011
www.asse.org
lators recognize that several factors increase the
success of an organization’s safety process. One is
management’s visible support of the process (Bailey, 1997). Using a tool such as an employee safety
perception survey and reacting to the results in a
visible, positive manner sends a powerful message to the workforce, In addition, it has been suggested that the more employees are meaningfully
engaged in the SH&E process, the more successful
that process is, especially in relation to the number
and severity of injuries (Carder, 2003).
Using an employee safety perception survey,
the organization is tapping its best resource for
hazard identification, When the organization addresses identified issues in a positive manner, it is
attending to issues of immediate importance and
relevance to those at risk. PS
References
Bailey, C (1997,Aug.). Managerial factors related to
safetyprogram effectiveness:An update on the Minnesota Perception Survey. Professional Safety, 42(8),33-35.
Bailey, C.W. & Petersen, D. (1989,Feb.).Using
safety surveys to assess safety system effectiveness. Professional Safety, 34(2),22-26.
Blair, LH. & O’Toole, M.F. (2010,Aug.). Leading
measures: Enhancing safety climate and driving safety
performance. Professional Safety, 55(8),29-34.
Carder, B. (2003).A survey-based system for safety
measurement and improvement. Journal of Safeh) Research, 34(2), 157-163.
Cohen, J. (1992).A power primer. American Psychologist, 112(1),155-159.
Erickson, J.A. (1994).The effectof corporate culture
on injury and illness rates within the organization. Dissertation Abstracts International, 55(6).
Erickson, J.A, (2008,Nov.). Corporate culture:
Examiningits effectson safety performance. Professional
safety, 53(11), 35-38.
Fowler, F. (1995),Improving survey questions: Design
and evaluation. Thousand Oaks, CA:Sage Publications.
Ivers, R., Senserrick, T., Boufous, 5., et al. (2009,
Nov.). Novice drivers’ risky behavior, risk perception
and crash risk:Findings from the Drive Study. American
Journal of Public Health, 99(9),1638-1644.
Johnson, S. & Hall, A, (2005).The prediction of safe
lifting behavior: An application of the theory of planned
behavior. Journal of Safety Research, 36(1), 63-73.
O’Toole, M. (2002).The relationship between
employees’ perceptions of safety and organizational
culture. Journal of Safety Research, 33(2), 231-243.
O’Toole, M. & Nalbone, D.P. (2007).Is safety
climate a barometer of safety results? Proceedings of
ASSE’s
Safety 2007, Las Vegas, l’-.,TV,USA.
Peterson, D. (2005,[an.). Safetyimprovement: Perception surveys can reveal strengths and weaknesses.
Professional Safeh}, 50(1), 45-48.
Seo, D-C. (2005) An explicativemodel of unsafe
work behavior. Safety Science, 43(3), 187-211.
Zohar, D. (1980).Climate in industrial organizations:
Theoretical and applied implications. Journal of Applied
Psychology, 65(1),96-102.
Zohar, D. (2005).A multilevel model of safety
climate: Cross-level relationships between organization
and group-level climates. Journal of Applied Psychology,
90(4),616-628.
Safety Management
Dan Peterson
Process of change
• Three simple steps
• Determine where you are now
• Decide where you would like to be
• Provide the difference
New safety culture
• Safety is a people problem
• To achieve substantial breakthroughs in
accident prevention we must focus on people
– Line worker – to worried about security of job to
remember to test respirator
– Supervisor – concerned with production
schedules that time is not available to follow up
on training new employees
– Manager – pressured for immediate financial
results and decides to hire two new sales people
rather than invest in safer machinery
Upper management support
• Through their words and more importantly
through their actions, upper management sets
tone for safety in the organization.
• In a good safety culture, management
frequently expresses vocal support for safety by
taking positive visible actions that let each
employee know that safety is a key value.
• Upper management sets the tone
Active Middle Management
Involvement
• Middle managers must create a uniform set
of policies and procedures that support and
nurture the safety culture.
• There must be consistency throughout the
management team that leaves no doubt
about the seriousness of the commitment
Forced supervisor performance
• Strong management commitment deals with
the day-to-day activities at the supervisory
level.
• In positive safety culture supervisor is always
concerned with safe behavior of employees
• Major component of supervisors own
performance appraisal must focus specifically
on how well this job is being accomplished
High level of participation
• In good safety culture, each member of the
company participates fully in the various safety
activities.
• Safety is no longer thought of as an activity that
someone else is in charge of.
• Each manager, supervisor and line worker feels
personally responsible for safety and take an
active role in promoting it.
Program flexibility
• A good safety culture will allow each group
within a company to handle their safety
issues according to their own particular
needs and circumstances.
• The right way to achieve safety is the way that
works in your operation.
Positive perception among employees
• In good safety culture, employees will have a
positive perception about the various
activities.
• They will have confidence that safety is always
being addressed and that their voice will be
heard in making decisions relating to safety
Dan Peterson
Dan Peterson
Steps for change
• Assess your current safety culture
• Identify the needed changes
• Plan and implement specific changes
Role of Safety Professional
• Safety Professional will be involved in the entire process most of the activities will be managed and performed by
others.
• Safety Professional is facilitator or coach for people who
carry out the work.
• Facilitator is content expert for overall process, trainer for
the required tools and techniques, and motivator to keep
process moving.
• Being facilitator is quite different from being “in charge”
and while adjustment may require some effort, the rewards
can be enormous. Successful facilitators derive satisfaction
not form their individual accomplishments, but from creating
the opportunity for the entire company to function at peak
performance.
Perception Survey
• Predefined set of 100 questions design to
specifically measure how your employees
perceive the company’s performance in 21
safety categories
• Survey should create a graph showing which
categories employees perceive as being most in
need of improvement
• Survey should highlight different perceptions
held by managers, supervisors and employees
• Change process only works when all members of
the organization agree on the nature of the
safety problems to be addressed.
Perception Survey
• Safety categories that show markedly different
perception scores for the three groups are
areas where a renewed communications effort
will be extremely important.
Interview Method
• Alternate to perception survey – goal of both
techniques are the same
• One-one-one discussions are held with
number of employees to determine what
areas of the safety program need
improvement
• Interview method will deliver more
information, but its success is directly
dependent upon skill level of interviewer.
Change
• Identify the needed changes
– Identify specific actions that should be taken to
correct problem areas
– Approach is employee-based, problem-solving work
groups for each of the low rated safety categories
– Task to identify and recommend solutions
• Plan and implement specific changes
• Implement the specific recommendations
– Many will be simple and can be implemented as
soon as possible
– Others may require significant expenditure of funds
or demand coordination of large number of people
• In these cases necessary to put together written project plan
that describes and coodinate steps need for implementation
• “Everyone should know
that their answers to the
survey will remain
confidential and
anonymous”
Likert
•
•
•
•
•
•
The format of a typical five-level Likert item is:
Strongly disagree
Disagree
Neither agree nor disagree
Agree
Strongly agree
Peterson copyrighted
Peterson copyrighted
SCRAPE
• System of Counting and Rating Accident
Prevention Effort is purely a systematic method
of measuring accident prevention effort.
• Indicates amount of work done by supervisor
and by company to prevent accidents in a given
period
• Purpose to provide tool for management that
show, before accidents, whether positive means
are being used regularly to control losses
• System includes: inspections, investigation,
injury records
Inspection
• Most important reason for inspection is to
measure manager/supervisor performance
• If used as measurement tool supervisor/line
manger will conduct more inspections to
ensure conditions remain safe and that fewer
unsafe acts will occur
• Inspections preformed by staff professionals
should be for purpose of auditing supervisor
effectiveness and be a direct measurement of
supervisor performance
Investigation
• Primary accident investigation function has
always been the supervisor’s.
• Tools we give should lead to determination of
some of the many underlying cause
• Proper that line supervisor should investigate
and be allowed to determine what really
happened
• Form used should force identification of at
least five causes and encourage dealing with
all of them
Injury records
• Records should be designed so that they
measure the line manager, and to measure
the results of the line manager’s safety
performance
• Should be set up so that:
– Accident records are kept by the supervisor
(department)
– Give some insight as to how the accidents seem to
be happening (agency, body part, event, etc.)
– Conform to any legal and insurance requirements



Purchase answer to see full
attachment