..

George Teston, Ph.D. | 360 Degree Educational Assessment

banneraccent.JPG (1969 bytes)
MAIN MENU

website by
George Teston,
Ph.D.
email: George Teston



The 360o Model Meets the Educational Organization

Introduction to the Principles of Educational Organizations

Measuring Organizational Effectiveness in Business & Education

The 360o Model for Assessing Organization Effectiveness



We Want Your Feedback about the 360o Model!             A Metaphor for 360-Degrees Assessment

 

 

 

..

 

George Teston, Ph.D. | 360 Degree Educational Assessment

banneraccent.JPG (1969 bytes)
The 360o Model Meets
the Educational Organization
Why and how has it been used in education?

by George Teston

Iowa State’s 360-degree Feedback Model

        The 360-degree assessment model, already well established in business and industry, is slowly becoming a popular method for measuring organizational effectiveness within schools as well. While several 360-degree models such as Skillscope/Benchmarks and the Clark-Wilson instrument have been adapted from the business community for school use, one model was developed expressly for use in educational organizations. The School Improvement Model Projects Office of Iowa State University’s College of Education, under the direction of Dr. Richard Manatt, developed an evaluation tool based on the unique structure, environment, and desired outcomes of the K-12 school. The Iowa State 360-degree Feedback model offers public schools a fresh, multi-source assessment instrument that is more appropriate and accurate than traditional approaches (Manatt and Benway, 1998).

        Manatt (1997) states that there are basically two reasons why this model is better for school assessment: multiple criteria and evaluators yielding increased accountability and rating that, unlike traditional top-down methods, provides true discrimination of performance levels among similar employees (p.9). He adds that, "The overreaching purpose of performance evaluation is to improve performance year after year…It just doesn’t happen using the old, almost ceremonial approach" (p. 9). Less Shulman (as quoted by Manatt) found in his 1991 Carnegie project that, "nearly every method one can imagine for teacher performance evaluation is marred in a fundamental way." The Iowa State 360-degree Feedback model has been implemented in many districts across the nation. Indeed, the field record of the instrument seems to be more successful than traditional models.

A five-year study of 360-degree feedback in the Hot Springs County School District in Thermopolis, Wyo., identified a 15 percent increase in achievement across all subjects measured by the SRA standardized tests. These gains occurred over the period with no decline in morale among teachers and principals due to the ironclad accountability of 360-degree feedback. (Manatt, p. 97).

        The Iowa State 360-degree Feedback model uses multiple data sources and different approaches for different employees. Performance data is gathered from principals, peers, parents, students, self-reflection, and student achievement statistics. Instead of each teacher being evaluated by the same means as all the others, the Iowa State model has three "tracks" that determine the factors and means by which the employee will be assessed. The first track includes beginning teachers and offers strong support resources and basic training. The second track includes those teachers who do not meet the school’s performance objectives and who need assistance to reach that level of quality. The third track includes experienced teachers who are already meeting school expectations and provides each with direction for continued improvement. In addition to teacher evaluation, the model also assesses principal and superintendent performance. In fact, every working individual on the school staff is evaluated by all who have contact with that person: supervisors, peers, students, and the public (Manatt, p. 8).

        Six data sources are used in teacher evaluation: supervisors, self-evaluation, student achievement, peer feedback, student feedback, and parent feedback. Ten data sources are used in principal evaluation: supervisors, self-evaluation, student achievement, student feedback student attendance, retention levels, teacher performance data, teacher feedback, parent feedback, and school climate. The superintendent is evaluated by a competency survey from five data sources: board members, principals, curriculum specialists, teachers, and parents.

        The evaluation instrument used by these people is a performance appraisal survey that is specific to each evaluator audience. Students are surveyed in class with a twenty-item survey that is age-grade specific about the teacher. The questions address student interest, instructional delivery, and preparation for teaching. Peer teachers must be field-appropriate and evaluate using the same criteria as the students and the principal. Self-assessment is done by teachers completing a twenty-item survey similar to the one provided by students. A Principal’s evaluation of a teacher is beyond the scope of a survey, including observation ratings, interviews, work samples, and progress toward teacher-set improvement goals. In turn, the teachers evaluate principals based on a school climate survey. Parents are surveyed using a five item instrument at the conclusion of a parent-teacher conference and is submitted blindly.

        Manatt and Price found in a 1994 study that student achievement is the most significant performance indicator. In the Iowa State 360-degree Feedback Model student achievement data is collected for each class, subject, and grade. Each class has a pre-test and a post-test to measure learner gain during the period under the teacher’s instruction. The percentage of mastery indicates exactly how much progress a teacher’s students make while under his or her tutelage. Manatt states, "This data required several years of curriculum renewal, alignment, and assessment…it’s the ultimate 360-degree source" (p. 10).

        Field-ready surveys are available for purchase from the Iowa State School Improvement Model Project Office or the school can make its own copies of each survey using a master available for no charge. All surveys are administered anonymously and can be scored externally by the Iowa State SIMP office for a fee. Or if the school elects to do internal compiling, the Iowa State SIMP Office will provide survey analysis keys to the school for no cost whatsoever. The no-cost alternative provides even the most financially challenged district an opportunity for quality, comprehensive evaluation.

        When analyzed collectively, the survey instruments illustrate the areas of high competence and areas needing improvement. Teachers individually develop a professional growth plan while the organization itself develops an improvement plan. Everyone has a professional growth plan, even experienced teachers with stellar evaluations because a tenet of the Iowa State 360-degree Feedback is the need to establish "continual improvement." Manatt’s research has shown that the action plans are most effective when deadlines are negotiated and imposed and when the goals are announced publicly (p. 11). The individual’s pursuit and accomplishment of these articulated goals becomes part of the evaluation process for the next year. This cycle imposes accountability of each member of the organization to recognize areas of improvement and complete the process to fulfill that need. The model provides guidelines for such action plans and suggests questions such as:

     Longitudinal studies (of 360-degree Feedback) in two districts have shown dramatic improvement in student achievement and equity (Manatt and Kemis, 1997). Richard Santeusanio, Superintendent of the Danvers, Mass. Public Schools, (as quoted by Manatt, p. 12) enumerates seven advantages to using the Iowa State 360-degree Feedback model district wide.

        In summary, the research obtained from field implementation of the Iowa State 360-degree model suggests that multi-source feedback is better than traditional appraisal done by a principal or superintendent alone. The 360-degree process also seems to have a greater impact on participants than single source evaluation. The success of the model suggests that it is used correctly, the Iowa State 360-degree Feedback model is not only the best choice for evaluating the organizational effectiveness of a K-12 school, but it may also be a tremendous tool for school reform.

For Iowa State 360-degree Feedback Materials:
You can obtain sample instruments, norm group data, and assitsnace in processing from: School Improvement Model projects Office, Attn: 360-degree concultant, College of Education, N225 Lagamarcino Hall, Iowa State University, Aimes, Iowa 50011-3195, or call 515-294-5521.

References

Manatt, R. (1997). Feedback from 360 degrees. The school administrator, 54 (3)., p.8-13.

Mannatt, R. and Benway, M. (1999). Manuscript in press. ERS Spectrum.
    Educational Research Service. 2000 Claredon Boulevard.
    Arlington, VA 22201.

Manatt, R. and Kemis, M. (1997). 360-degree feedback: A new approach to
    evaluation. Principal, 77 (1)., p. 24-27.

Manatt, R. and Price, P. (1994). Five factor teacher performance evaluation
    for career ladder placement. Journal of Personnel Evaluation, 8. p. 239-250.

 

Return to the Main Menu

 

 

 

 

 

 

 

 

 

George Teston, Ph.D. | 360 Degree Educational Assessment

banneraccent.JPG (1969 bytes)
Introduction to the Principles of
Educational Organizations
What is "organizational effectiveness?"

by Jacqueline Makins

Organizational Effectiveness

When working in an organization, we want to know the most competent people working in that organization and the behaviors of the people we think are incompetent and how they affect productivity. One way that we can find answers is by using tools that give us feedback about our strengths and weaknesses.

Robert Owens states in his book (Organizational Behaviors) that an organization has dependent variables that are indicators of the organizational effectiveness. Dependent variables measure achievement, behaviors, and school effectiveness (1998, p. 179). Traditionally, organizational measurement has been used to monitor and control employee behavior. Rarely has it been used to provide employees with feedback on performance, to reinforce good performance, or as a tool for helping employees to solve problems (Boyett and Conn, 1991, p. 65). One tool in particular is the 360-degree model used for many different purposes. One purpose is to address the different roles in an organization and to enhance and maximize the organization and individual development. Common uses are for development and training, performance appraisal and performance management, and as a culture and attitude survey within an organization. It can be delivered in a variety of formats: from informal questionnaires to interviews and surveys that measure skills and compare the ratings to normative populations. Development needs are also identified, making appropriate training and action plans easier to develop in order to initiate positive change. The profiles stimulate greater commitment to personal growth and help to develop more effective leadership and management skills.

The 360-degree assessment, that are used in educational organizations, are effective in helping an employee not only see what he or she does well or needs to improve, but also helps create a powerful developmental plan that can reduce years from the trial-and-error method of personal improvement. Measurement professionals generally agree that for assessments to be valid, they should (a) measure the construct they are intended to measure, (b) measure the content taught, (c) predict students' performance on subsequent assessments, and (d) provide information that is consistent with other related sources of information (Nolen and Taylor, 1996).

The 360-degree assessment also forms a picture of the strengths and weaknesses of an organization and its readiness to face marketplace and industry changes. In addition, the assessment serves as an outstanding foundation for developing a training strategy for the organization. The National PTA states that an effective school should use resources to respond to educational challenges and assume responsibility for investigating quality programs such as 1) curriculum, 2) opportunities for professional development, and 3) safe and well-maintained school buildings, just to name a few (National PTA, 1998 web site).

The National PTA recommends that schools should use "sound" assessment programs that are:

Owens identified several different tools used for assessing organizational climates but these perceptions (some people argued) are not "objective reflections" of reality and may be subjective in nature. One assessment described teacher’s perception of other teachers. Another assessment is a collective perception of teacher’s perception of the principal (p. 182-183). Together both of these perceptions make up the tools used to identify the effectiveness of an organization.

In educational organizations, effectiveness is built around the dimensions in the feedback report. If the feedback is good, then the organization is doing what it should in order to make that organization an effective organization. If the feedback shows that the organization needs improvement based several studies and research, then the organization must provide special remedial and other instructional support in order for the organization to be effective. Also effective organization places high expectations on the people within the organization because the belief is that:

"No organization can succeed unless the individuals within it have the freedom to be themselves" (Kolbe, 1990, p. 97).

What Kolbe suggests is that each individual has to have the freedom to contribute their "gifts and talents" to the organization in order to make that organization what it should be. Sometimes employees do not have a voice in the decisions that are made. They feel "left-out-of-the-loop" and want to contribute their ideas and suggestions. One way to get suggestions from employees is to administer one of the 360-degree models and see what the feedback report details. Most 360-degree models do not require any certification programs and is easy for participants to understand. Most models take about 30 minutes to complete and are an assessment of job-related skills.

This web site will show how different models of the 360-degree assessments are used in organizations. Whether you work in an educational setting or business, hopefully you will decide on the tool that is right for your organization. Owens states that "many studies have described relationships between performance of companies in terms as market share, sales, and profitability and the organizational culture within the company." Evidence exist from research in educational settings that effectiveness is determined by "student learning and development" and is also "influenced by the quality and characteristics of the organizational culture" (p. 193). But the idea of organizational effectiveness is to find out "who is competent and who is not" and to see what those employees do well or where they need to improve.

 

References

Boyett, Joseph and Conn, Henry. (1991). Workplace 2000: The Revolution Reshaping American Business. New York, NY: Penquin Group.

Kolbe, Kathy. (1990). The Conative Connection: Uncovering the Link between Who You Are and How You Perform. Menlo, CA: Addison-Wesley Publishing Company, Inc.

National PTA. (1998). Components of an Effective School. URL: http://www.pta.org/programs/effschl2.htm#Assess

Nolen, Susan and Taylor, Catherine. (1996). What Does the Psychometrician's Classroom Look Like?: Reframing Assessment Concepts in the Context of Learning. Education Policy Analysis, 4-17.

Owens, Robert G. (1998). Organizational Behavior in Education. Needham Heights, MA: Allyn & Bacon.

 

Return to the Main Menu

 

 

 

 

 

 

 

 

 

..

George Teston, Ph.D. | 360 Degree Educational Assessment

banneraccent.JPG (1969 bytes)
Measuring Organizational
Effectiveness in Business & Education
What and how to you assess?

by Kimberly Reely

Measuring Organizational Effectiveness

        There is a need for balance and trade-off to achieve organizational effectiveness. The establishment of goals are important. The culture and climate of the organization have a great impact on how effective an organization will be. And the historical performance of an organization must be analyzed for baseline data. Organizational effectiveness is quantitative and can be measured.

        In the Robert G. Owens text, Organizational Behavior in Education, (p 57) the author discusses a study by Chester Barnard and discusses ‘equilibrium as “the balancing of burdens by satisfactions which results in continuance” of both the individual and the organization in a mutual relationship. In his lexicon, the term “effectiveness” was the “accomplishment of the recognized objectives of cooperative action.” Effectiveness must represent a shared vision. The achievement of goals is how effectiveness is measured. The complicated part is how to motivate the individual organizations and ultimately measure the
outcomes.

        Many times the effectiveness of an institution is measured during a time of fundamental change, increased competition or crisis. Effectiveness of organizations is difficult to understand and complex. In an article published in the journal of Research in Higher Education, titled Maintaining Effectiveness Amid Downsizing and Decline in Institutions of Higher Education,, Kim Cameron and John Smart mention how difficult it has been traditionally to measure effectiveness in organizations. In fact, they state (p 69) ‘Traditionally, measuring effectiveness has been so confusing that some scholars have advocated abandoning the concept .” They go on to mention historically “In higher education, the dominant approach for many years was to rely on a single overall assessment such as reputation ratings.”

        Through the years management techniques have changed. “More recently, assessments in the scholarly literature have begun to take into account a much more complex view of higher education institutions, including considerations of multiple constituencies, environmental contingencies, and the paradoxical nature of organizational performance.” (Cameron, p 69). This current mode of study reinforces our current recommendation for a 360 measurement tool to determine organizational effectiveness. The term 360 uses the assumption that organizations (and even individual performance evaluations) are measured from multiple sources that include top down and bottom up sources. Often times the measurement tools are perception surveys that are distributed to different constituents within an organization. It may include customer satisfaction, employee rating, external agencies, etc. Then the results are compiled to form a composite measurement of effectiveness.

        The 360 assessment tool is similar to classroom assessment which uses multiple measures to determine acquired knowledge by students. Even though the measurement tools are designed differently, the strategyis the same. In the classroom, multiple measures are used to determine knowledge is various formats. Assessment could be in the form of multiple choice responses, short answers, essay writing, portfolio anddialogues. These are examples of multiple measures used in a learning environment.

        Organizational effectiveness is different from a classroom because what is being measured in an organization is the outcomes of a collection of people and how the organization is meeting the end statements. Cameron and Smart list in their study what they consider to be the “Nine Dimensions of Organizational Effectiveness” (p 70). They are:

        Morale Domain:

                Student Educational Satisfaction: The extent of students’ satisfaction with their educational
                        experiences at the institution.

                Faculty and Administrator Employment Satisfaction: The satisfaction of faculty members and administrators
                            with their employment and jobs at the institution.

                Organizational Health: The extent of smooth functioning of the
                            institution in terms of its processes and operations.

            Academic Domain:

                Student Academic Development: The extent of academic attainment, growth, and progress made by
                            students at the institution, and the opportunities provided by the institution.

                Professional Development and Quality of Faculty: The extent of professional attainment and
                            development of the faculty, and the amount of stimulation toward professional development
                            provided by the institution.

                Student Personal Development: The extent of development socially, culturally, and emotionally,
                            and the opportunities provided by the institution for personal development.

            External Adaptation:

                Student Career Development: The extent of occupational or vocational development of
                            students and the opportunities provided for occupational development by the institution.

                System Openness and Community Interaction: The emphasis placed on the interaction with,
                            adaptation to, and service in the institution’s external environment.

                Ability to Acquire Resources: The extent to which the institution acquires resources from the
                            external environment such as economic support, high-quality students and faculty, research
                            support, and political legitimacy. (p 70)

        As you can see these measurement tools are very comprehensive and acquired from multiple constituents. These specific measurement guidelines are used for ‘traditional’ institutions of higher education. Even in programs offered online, the measurement indexes must be designed specifically for the institution because the programs offered online are ‘non-traditional’ by nature. These domains represent the type of shared vision institutions must agree upon prior to conducting an analysis of institutional effectiveness.

        One other item that must be used in measurement studies is the period of time used for evaluation. It may be every six months; the end of each fiscal or calendar year; and then comparisons from year to year. The feedback and program evaluation is necessary to see if positive or negative change has occurred.

        One other notable contribution from the study by Cameron and Smart (p 72) was the presence of attributes they named the ‘dirty dozen’. These attributes are present to some degree in most organizations, but how they are addressed by managers and administers determines the impact they will have on organizational effectiveness. Here is the list of 'dirty dozen'.

        (A complete review of this article can be found by accessing http://www.catchword.com/). The article will go into greater detail  what each of these attributes represents within an organization.

1. Centralization
2. Short-term crisis mentality
3. Loss of innovativeness
4. Resistance to change
5. Decreasing morale
6. Politicized interest groups.
7. Nonprioritized cutbacks
8. Loss of trust
9. Increasing conflict
10. Restricted communication
11. Lack of teamwork
12. Scapegoating leaders

        The outcomes of this study indicated the ‘Dirty Dozen’ did impact organizational effectiveness, but not in all cases. An organization could still be effective, even in a period of decline if the ‘dirty dozen’ attributes are kept in check. According the Cameron and Smart study (p 79), “In other words, analysis of the individual dirty dozen attributes produces support for the conclusion that when institutions allow themselves to develop these negative attributes, institutional performance suffers.” In addition the article mentions “--leadership actions and institutional member reactions--help explain why institutions experiencing decline demonstrate a range of performance levels.”

        This conclusion if very important to the overall theory of organizational effectiveness. We can use all the organizational effectiveness measuring tools we choose, and promote professional growth and evaluate desired outcomes, but if the leadership of the organization  is weak, misdirected or without a clear mission, the organization will falter.


REFERENCES

Cameron, K and Smart, J, Maintaining Effectiveness Amid Downsizing and Decline in Institutions of Higher Education, Research In Higher Education, February 1998, Human Sciences Press, New York, NY

Owens, R., Organizational Behavior in Education, 1998, Allyn and Bacon, Needham Heights, MA

 

Return to the Main Menu

 

 

 

 

 

 

 

 

..

George Teston, Ph.D. | 360 Degree Educational Assessment

banneraccent.JPG (1969 bytes)
The 360o Model for Assessing
Organization Effectiveness
What is this process and its origins?

by Nancee Simonson

The 360 Degree Assessment Tool as One measure of Organizational Effectiveness


Assessing an organization's "current state" is one of the best ways to determine what is working and what needs improvement. How effective is your organization's leadership? What issues are currently affecting work force morale? Are financial goals well understood throughout the organization? Assessment models to answer these and other critical organizational performance questions include:

University of Santa Bar Barabara Extension customized education program in Organizational Effectiveness, http://www.unex.ucsb.edu/programs/ce/oea/

Paula K. Martin and David Huckle provide organizational effectiveness tools in four different areas:

  1. Processes & Projects
  2. Organizational Structure
  3. Management Practices
  4. Organizational Goals

http://www.1res.com/RENMGMT/OE/OEHOME.HTML
 

Three models of 360 degrees assessment

This section focuses on the 360 degree assessment tool and various models of that tool as one choice for assessing leadership in an organization. The next section will focus directly on the tool as it has been applied in exemplaries ways specifically within educational organization. The conceptof 360 degree assessment basically involves review or evaluation of a leader, manager or employee by various levels of people with whom the person has contact. This includes superiors, co-workers, reports, and in the case of schools may include parents or community members. This concept departs from the traditional top-down hierarchical method of reviewing and evaluating employees and incorporates a systems view of performance in that it acknowledges that performances affects stakeholders from many levels of the organization.

Three models described below are:

  1. ManagerView/360; PerformanceView/360
  2. Skillscope/Benchmarks
  3. Clark-Wilson Group instruments

The instruments organize evaluation criteria into various categories of factors. These factors need to be based on the organizations goals and concepts of quality or performance. (Note that one organization offers customized  evaluation instruments.
 

1. ManagerView/360; PerformanceView/360

Manager View/360

Manager View/360 compares one's perception with those of peers, direct reports and one's manager; and provides an objective summary of an employee's strengths and areas of development along twenty competencies determined to be required for competitive performance. It is used  for supervisory training, management development, training needs assessment, career development, and training evaluation. http://www.opd.net/ps-p-mv360.html

ManagerView/360 scales include:

Communication Competencies

Task Management Comptencies

Interpersonal Competencies

 

For project leaders:

 

Return to the Main Menu

 

 

 

 

 

 

 

..

 

 

 

 

..

 

 

 

 

 

 

 

 

 

 

..

George Teston, Ph.D. | 360 Degree Educational Assessment

banneraccent.JPG (1969 bytes)
A Metaphor for 360-Degrees
Our project experience convinced us that::

 

Effective assessment of schools and the people within it is more than a tool; it is a journey.
It is not a product, but a process. That process should, like learning itself, be multidimensional, mastery-oriented, improvement-based, life-long, equitable, personally relevant, and applicable.

 

 

Return to the Main Menu