|
|
|
Evaluation of Staff-Development Efforts to Impact by Pamela Tipton, Ed.D., Director of Instructional Programs Abstract Data collected on staff-development opportunities for technology proficiency and its impact in the Roswell Independent School District document that (1) the Roswell school system is providing more occurrences of workshops and other training and (2) the teachers are participating in more staff development on technology. The increase in the number of offerings and participation is summarized in the following table.
1992-1993 1993-1994 1994-1995
Number of
workshops, 10 35 195
training sessions
and supported
conferences
Number of teachers
participating in 195 370 1679
the available
session
Resulting number
of 1993 4415 6826
staff-development
hours educators
spent learning
technology
The results have been (1) an increase in technology use by teachers and their students, (2) an increase in the awareness of technology planning, (3) a belief that the resources necessary for improved proficiency with technology are available, and (4) self-reported proficiency levels have surpassed the goal of the board-adopted technology plan. The staff-development processes and outcomes, along with factors driving and impeding technology integration in the classroom, are the basis for recommended modifications to practice and policy for an improved plan. I. What staff-development processes have been enacted to contribute to the acceptance and use of technology? Initiatives undertaken by the Roswell school district from January 1994 through June 1995 to provide teachers with the skills and support to become proficient personal users of technology and to integrate technology in their curriculum have focused on people, processes, practice and policy. Interventions centered on people, processes and practice include:
Policy implementation has included establishing building inservice time, entering into a school-community partnership for the Model Technology School, and initiating a process for site planning for technology implementation and technology expenditures. A chronological accounting of staff-development participation at district-sponsored training and conference support since January 1994 follows.
Workshop Date & Time Workshop Title Number of Total
Participants Educator
Hours
February 8 - April 26, Technology for the 19 494
1994 Instructional Leader
Tuesdays, 4:00-6:00 p.m.
January 18, 2026 SUPERPRINT for the 10 30
1:00 - 4:00 Language Arts & Science
Curriculum
March 31, 2026 Introduction to the 2 4
1:00 - 4:00 Internet, NMMI
March 25, 2026 TECHNET Training 26 78
1:00 - 4:00
April 4-5, 1994 Multimedia Background, 37 444
8:00 - 3:00 p.m. Basics and Applications
April 6-7, 1994 Multimedia Background, 39 468
8:00 - 3:00 p.m. Basics and Applications
April 13, 2026 ETI Conference, 1 6
Albuquerque
June 6-10, 1994 Introduction to 27 540
8:00 - 12:00 Integrated Software:
ClarisWorks
June 6-10, 1994 Introduction to 26 520
1:00 - 5:00 Integrated Software:
ClarisWorks
June 13-16, 1994 Introduction to 19 456
9:00 - 4:00 Integrated Software:
Microsoft Works (DOS)
June 17, 2025 Introduction to HyperCard 19 57
9:00 - 12:00
June 20-24, 1994 Introduction to 18 360
8:00 - 12:00 Integrated Software:
ClarisWorks
July 12 - 16, 1994 MacAcademy/Learning 2 72
Extravaganza
August 2-3, 1994 Followup Multimedia 33 230
8:00 - 4:00 Workshops
August 1 - 11, 1994 IBM TLC for the Model 25 1400
8:00 - 4:00 Technology School
August 8-9, 1994 Southeastern Technology 1 12
Conference, Carlsbad
September 1994 - May 140 sessions on various 715 1430 (est)
1995 topics at the Technology
1-3 hour sessions Support Center
November 3-4, 1994 EduQuest software 20 240
training
November 23, 2025 Building Inservice Day 300 (est)
8:30 - 3:30 p.m. Chisum: Josten's Lab 14
EGP: Computer Software 16
Review 18
MilH: Computer Training, 20
Update 16
MoAve: Software review
Pecos: Hands-On Computer 16
Training 11
VV: Software Review
UHS: PLATO software
October 17-19, 1994 Technology Tour to 3 72
California
November 7-8, 1994 Southern New Mexico 7 70
Technology Conference,
Las Cruces
November 10-12, 1994 National TEL*ED 6 120
Conference, Albuquerque
January 3, 2026 Building Inservice Day 460 (est)
8:30 - 3:30 p.m. BES: Multimedia 23
Chisum: Josten's "Steps" 14
ESL Program
DelNorte: Calculators in 20
the Classroom
El Capitan: Multimedia 16
MoAve: Multimedia 20
WashAve: Technet - 27
Telecommunications
MtV: Technology in the 27
Classroom
SMS: ClarisWorks & 25
Josten's Learning System
UHS: PLATO workshop 11
January 21, 2026 Teaching and Learning 154 308 (est)
8:00 - 1:00 With Technology: A
Technology Fair
February 17, 2026 Building Inservice Day 790 (est)
8:30 - 3:30 BES: Technology 23
Chisum: Multimedia Center 14
Del Norte: Math & grading 20
software
EGP: Software Review 16
ElCap: Computer Literacy 16
Pecos: Technology 16
BMS: Multimedia Center 25
MMS: Technology 29
MilH: Computer Usage 18
Mont: Language Arts & 29
Technology
Park: Technology 14
SMS: Technology 25
GHS: Technology Planning 70
March 14-15, 1995 Technology Tour to 4 48
Colorado
March 20, 2026 Apple's Best of All 10 60
Worlds
Conference, Albuquerque
March 24-26, 1995 Educational Technology 3 45
Institute, Clovis
March 28, 2026 Community Technology 4 24
Field Trip to Hobbs &
Lovington
May 2-5, 1995 IBM's Technology & the 3 90
School Executive
Conference, New York
May 25, 2025 IBM Internet Servers & 8 16
10:00 - 12:00 WAN Management
June 14-18, 1995 National Educational 1 40
Computing Conference,
Baltimore
June-July, 1995 Online Internet 2 64
(four onsite days) Institute, Albuquerque
May 30 - June 2, 2025 ClarisWorks 11 165
8:30 - 11:30
May 30 - June 2, 2025 Information Superhighway 12 180
1:00 - 4:00
June 5 - 9, 1995 Tapping the Power of 10 150
8:30 - 11:30 Technology
June 5 - 9, 1995 Microsoft Works (DOS) 5 75
1:00 - 4:00
June 12 - 14, 1995 ClarisWorks 10 90
8:30 - 11:30
June 14 - 16, 1995 ClarisWorks 5 45
1:00 - 4:00
June 20 - 23, 1995 Information Superhighway 13 156
8:30 - 11:30
June 20 - 23, 1995 Using Multimedia 8 96
1:00 - 4:00
Totals 205 sessions/workshops 1924 10,323
and/or participants educator-hours
conferences
The Technology Resource Team is a supportive network of school-based technology leaders. Twenty-five teachers, representing twenty-two schools, have served on the Technology Resource Team since February 1994. Part of the member's responsibilities include providing support to staff members at their school site who are integrating technology into the curriculum (one-to-one support, informal training, and formal workshops). Twenty-one participants responded to a survey in March 1995 and reported time spent on various aspects of their responsibilities. The amount of time reported on support activities include:
II. How effective have those processes been? Staff who attended the Technology Support Center and completed evaluation forms reported being at opposite ends of the continuum in terms of personal technology use. Although 23.1% of the 316 respondents reported over 20 hours of computer use for personal or professional reasons, another 31.0% reported using the computer less than two hours a week. When the results of the current self-reported computer use is noted alongside the self-reported results of the 1993 technology survey, there has been an increase of reported computer use, as illustrated in Table 1. Since the participants of these two surveys represent overlapping groups, a comparison cannot be made. Table 1. Self-reported Personal/Profession Use of Computers
Five or less hours per More than 10 hours per
week week
Source: Evaluation
forms from the 58.5% 27.7%
Technology Support
Center, 1994-1995; N =
316
Source: District-wide
technology survey, 81.4% 6.1%
September 1993; N = 425
Table 2 portrays that teachers reported similar changes when asked about student use of technology.Table 2. Student Use of Computers/Other Technology Reported by Teachers
Two or less hours per More than four hours
week per week
Source: Evaluation
forms from the 28.5% 71.5%
Technology Support
Center, 1994-1995; N =
316
Source: District-wide
technology survey, 70.8% 7.3%
September 1993; N = 425
Teachers who attended the Technology Support Center were also asked the effectiveness of various staff-development efforts for technology. The 316 respondents provided the following information, summarized in table 3. Table 3. Overall Evaluation of Staff-Development Activities
Effectiveness of
Staff-Development Effort Number who Number who staff-development
had had effort
participated not (1=low to 7=high)
participated (mean score
reported)
Summer workshop (prior to 66 249 5.29
1995) on ClarisWorks or
Microsoft Works
Summer workshop (prior to 19 297 5.20
1995) on multimedia or
HyperCard
One computer classroom 87 228 5.31
workshop
1-1 help from technology 178 135 5.84
person at my building
Class/workshop from 140 174 5.67
technology person at my
building
Vendor-presented workshop 81 235 5.50
(for example, Josten's,
Writing to Read, Writing
to Write, etc.)
Technology course for 30 285 5.50
credit
Watching video 26 289 5.12
Reading journals 18 296 4.06
Attending conference 12 302 5.36
Technology Support Center 316 n/a 6.50
Although all staff-development efforts for technology were rated as effective, this specialized group of teachers (those who attended the Technology Support Center) rated the Center as most effective. A district-wide technology survey conducted in May 1995 found that 77.5% of the 204 respondents perceived that they have access to the resources needed to become more proficient in the use of technology. When asked to self-report on their proficiency with technology, 79.9% of the 204 respondents believed they were proficient at the personal/professional use level, which includes ability to use word processing programs, maintain and access student records electronically, and use tool software to develop banners, posters, fliers, newsletters, and other daily tasks. This compares favorably to the September 1993 Technology Survey where only 43% of the 425 respondents believed they did not need support at the personal/professional use level. Almost seventy percent (69.1%) of the respondents indicated proficiency at the beginning instructional use level, which includes the ability to use a single computer for small- and large-group instruction, to use computers to enhance writing skills across the curriculum, to match software to curriculum content and needs of students, and to manage the classroom for effective integration of computers into the curriculum. Twenty-two percent (22.1%) perceived proficiency at the advanced instructional use level, which includes use of desktop publishing, spreadsheets, and graphing programs for professional use, simulations and other instructional software to develop critical thinking and problem-solving skills, and the ability to integrate the use of databases and spreadsheets into the curriculum. Twenty (20.1%) percent indicated proficiency with multimedia. Eighteen percent (18.1%) indicated proficiency with telecommunications. In May 1993, there was only one educator in Roswell with a TECHNET account, the state's Internet access provider. Today, 120 of the approximately 800 teachers in the district have TECHNET accounts. Respondents also reported awareness of district and school-based technology planning. Almost 95% (94.4%) reported that they were knowledgeable about their school's technology plan, while 88.7% indicated that they were knowledgeable about the district's technology plan. The technology survey also indicates that the staff-development efforts have exceeded the Roswell Long-Range Technology Plan's goal of staff demonstrating proficiency in the use of technology, at least with the 204 respondents. The table below describes the targeted percentages of proficiency at each level and the self-reported results of the survey. Table 4. Comparison of Targeted Technology Proficiency with Actual Proficiency, May 1995
Technology Proficiency Level Technology Plan % Reported
Timeline Proficiency
for Proficiency May 1995
N = 204
1994-1995: 50%
Level 1: Personal/Professional Use 1995-1996: 75% 79.9%
1996-1997: 95%
1997-1998: 100%
1994-1995: 40%
Level 2: Beginning Instructional 1995-1996: 60% 69.1%
Use 1996-1997: 80%
1997-1998: 95%
1994-1995: 20%
Level 3: Advanced Instructional 1995-1996: 30% 22.1%
Use 1996-1997: 50%
1994-1995: 10%
Level 4: Multimedia Use 1995-1996: 20% 20.1%
1996-1997: 30%
1997-1998: 40%
Level 5: Telecommunications Use 1996-1997: 15%
1997-1998: 30% 18.1%
III. Analysis and Recommendations Data accumulated primarily through district-wide technology surveys conducted in September 1993 and May 1995 and staff-development evaluation forms indicate that both teachers' and students' use of technology are increasing, and teachers' self-reported level of technology proficiency is increasing. More important, teachers are aware of both building and district technology plans and believe that they have access to the resources needed to become more proficient with technology. During in-depth interviews with fourteen Roswell educators, factors influencing technology integration were named. Factors that encourage and support the use of technology include:
The barriers to effective use of technology mentioned by Roswell educators include:
Theoretical frameworks serve as lenses through which to view influencing factors on technology use and staff-development efforts to impact technology acceptance and use. The factors influencing technology are categorized as societal, financial, pedagogical, or personal forces for easier analysis. The factors are then examined from the Concerns-Based Adoption Model (Hord et al., 1987) framework. The change model of Knoster (1993) provides insight to the overall staff-development plan of the district. Roswell educators identified the factors that contribute to their acceptance and use of technology and factors that serve as barriers to that acceptance and use. These factors can be viewed as representing broader categories: societal, financial, pedagogical and personal forces (see Table 5). From a numerical view, factors that fall into the pedagogical, or teaching and learning, category appear to have more influence in the use of technology than factors falling into the societal, financial, or personal categories. Since the participants were not asked to rank the factors influencing their use of technology, this cannot be determined. Table 5. Categories of Factors Influencing Technology Use
Category of Driving Forces Barriers
Factors
Societal Community expectation Speed of technology
Skills for economic success development
Resources provided by two mil Access Issues of lack of
Financial levy, bond issue, state hardware, software,
legislature, operational network infrastructure,
budget and maintenance
breakdowns
Student Success through Traditional school
a) meeting individual needs culture that leads to:
b) offering involved and a) comfort with stasis
relevant curriculum b) classroom-management
c) increasing student concerns
Pedagogical motivation c) competing priorities
and initiatives
Instructional and management
tools for teachers Lack of
instructional-content
Teacher success bridges in software
Opportunities for personal Comfort issues of fear,
and professional growth inexperience or lack of
Personal training, and time
Support through
administrative vision and Additional
staff-development responsibilities
opportunities
The stages of concern dimension of the Concerns-Based Adoption Model (CBAM) is a tool to facilitate implementation of change. It can be used to help better understand the concerns that teachers using technology may have and strategies that may be employed to address those concerns. Seven kinds of concerns are identified and fall into three broad categories or dimensions: self, task, and impact. The nature of concerns is "developmental and interactive" rather than "inherently good or bad" (Hord et al., 1987, p. 43). Concerns are influenced by the participants' feelings about an innovation and their perception of support and assistance to implement change. After identifying concerns, interventions to respond to those concerns can be delivered in order to assist movement through the stages of concerns and to higher levels of use. The concerns expressed by the participants in this study fall primarily into stages two (personal) and three (management) of the stages of concern (see Table 6). Hord et al. (1987) states that only after personal and management concerns are reduced will impact concerns surface. This explains why the teachers are not more concerned about students' success and well-being. It also explains why teachers do not respond to attempts to persuade them to use technology to change the teaching and learning processes. Specifically the concerns expressed and corresponding levels are as follows: Table 6. CBAM Classification of Concerns
Dimen- Stage of Concern Specific Concern Expressed
sion
0 - Awareness
1 - Informational Lack of knowledge about hardware, software
Self and management strategies for grouping
Lack of teacher training
2 - Personal Lack of time to explore hardware and
software
Lack of time to visit with other teachers
about what works and what doesn't
Lack of access to hardware and software
Logistics of managing a One Computer
Task 3 - Management Classroom
Logistics of managing 5 computers and 20-30
students
Integrating software into curricular goals
"Covering" all curricular content that
needs to be addressed
Logistics of hardware obsolescence and
maintenance
Conflicting priorities of curricular
4 - Consequence activities
Impact Philosophical disagreement whether a
student-centered classroom or a
teacher-centered classroom provides the
best learning environment for students
5 - Collaboration
6 - Refocusing
Knoster's conceptualization of the change process as described by Wood (1995) is based on five requirements: (1) a vision or mission driving the change; (2) the training and skills necessary to implement the change; (3) motivational incentives; (4) resources, including time; and (5) action plans that guide the steps of the implementation (see Table 7). Table 7. Antecedents to Successful Change (Knoster, 1993, as reported in Wood, 1995)
Results of
Ingredients the
of implementation
Change
Vision Skills Incentives Resources Action Plan Change
Skills Incentives Resources Action Plan Confusion
Vision Incentives Resources Action Plan Anxiety
Vision Skills Resources Action Plan Resistance
Vision Skills Incentives Action Plan Frustration
Vision Skills Incentives Resources Treadmill
The Roswell Long-Range Technology Plan provides a vision and action plan for technology use in the district. Each school has been charged with determining its own technology plan that is detailed and consistent with the district plan. The Technology for the Instructional Leader course was designed to assist with that process. Support for site vision and planning was also supported through the Technology Fair. A continuing need in these areas is "buy-in" by all staff. Although the technology plan was developed with input from each building and has been distributed to all staff, some claim not to be aware of the plan or to disagree with it. The vision and action plan aspects of the Long-Range Technology Plan address Knoster's first and fifth elements, eliminating the confusion and treadmill results. The skills element is a major hurdle being aggressively pursued. The structure to provide skills building is in place with the Technology Resource Team representative at each building and the Technology Support Center, with a supportive resource teacher. Although participation at the Technology Support Center was good, most buildings do not use all of their "allocated" substitute days. The perceived need for skills enrichment must be enhanced for all teachers to "reach out" to available training. Technology Support Center evaluation form data indicate that the majority of teachers taking advantage of the Technology Support Center classify themselves at the beginning instructional level. Perhaps this is due to the nature of the program. Incentives by means of stipends for summer training and release time during the school day have been built into the staff-development program. The more important incentive, local building principal support and encouragement, varies building by building. For some individuals, a very motivational incentive is success and professional growth. Individual personality and style determines whether successful teaching and professional growth is an incentive. The lack of incentives result in resistance, according to Knoster. Since the definition of incentive is internally determined for each individual; this explains why there is resistance in some, but not all, teachers. The literature on characteristics of effective staff development provides additional insight on the effectiveness of the Roswell staff-development process, parallel to Knoster's elements of skills and incentives. The following table demonstrates the staff-development structure that meets the suggested characteristic. Table 8. Comparison of Staff-Development with Recommended Characteristics
Characteristic of Effective Roswell Staff-Development
Staff Development Structure
Programs are focused on an individual Technology Resource Team
school-site and linked to school-wide Site Planning Process
efforts.
Teachers are actively and Technology Resource Team
collaboratively involved, with a focus Site Planning Process
on a sharing of knowledge among
educators and on building teachers-
communities of practice.
The emphasis is on self-instruction, Technology Support Center
with differentiated learning Conference Support
opportunities.
Methods employed include demonstration Technology Support Center
(modeling), supervised trials (coaching) Workshops and Courses
and feedback (collective problem Technology Resource Team
solving), which provide concrete,
sustained, and ongoing training over
time.
Ongoing assistance and support is Technology Resource Team
available. Technology Support Center
Fullan (1991) warns, "No matter how much advance staff development occurs, it is when people actually try to implement new approaches and reforms that they have the most specific concerns and doubts" (p. 85). The Roswell staff-development structures attempt to provide the needed ongoing assistance and support onsite and at the district-level. The fourth ingredient for successful change, resources, is being aggressively pursued through community support of local funding, through grants, and through the site planning process. However, the need is great and the supply is limited. Using Knoster's model, this analysis would imply that individuals may experience anxiety because of lack of training, resistance because the "right" incentive has not yet been discovered for them, or frustration because of lack of hardware and software. Recommendations Specific strategies to enhance the technology-integration efforts can be initiated within the current policy and framework to address the concerns Roswell teachers have in implementing technology, as recommended by Hord et al. (1987). In addition, policy revision through the annual update to Technology for Teaching and Learning: A Strategic Plan will reflect insights gained through the analysis of factors influencing technology use and effective staff-development designs. Within current policy and practice, concerns can be met through:
The factors influencing technology use in the Roswell School District were presented to members of the Technology Resource Team on May 25, 1995. This committee analyzed the current staff-development process, given the new understandings from the participants of the study. Recommended changes to the technology staff-development endeavors include: * Instituting a regularly scheduled early-release day for technology assistance * Developing and offering a credit course on classroom management for small-group instruction, "How to Structure and Manage a Technology Classroom" * Establishing a software evaluation site at the Technology Support Center * Instituting additional means of sharing information about technology, such as a technology newsletter, a library of videotaped lessons utilizing technology, and encouraging classroom observations through the talent bank process. These committee recommendations will be the basis of the technology policy change, the annual update to Technology for Teaching and Learning: A Strategic Plan. Because the study was a short-term study confined within the period of January 1994 to June 1995, the long-term effects of the staff-development processes are unknown. A follow-up study to determine the extent to which technology use has been institutionalized in the Roswell district needs to be done. Although an assumption underpinning this research was that teachers' acceptance and use of technology will improve the teaching and learning processes, that belief needs further research. Questions, beyond the scope of this project, to be answered in future research include: Does the use of technology actually change teaching; and, if so, under what circumstances? Is student learning impacted by the integration of technology into the curriculum and instruction?
Return to RISD Special Programs
Support
Our Sponsors
|
