Introduction
In the last decade, executive coaching combined with 360 feedback has been one of the fastest growing executive development options within global companies (Goldsmith et al., 2000; Judge and Cowell, 1997) . It has even become``trendy'' to have one's own executive coach ± similar to having a personal trainer. Businesses invest large sums of money in matching executives with the right coach, yet, to date, there has been little empirical research on the impact of coaching on leadership effectiveness and payback to the organization (Kilburg, 2000; Hall et al., 1999) .
Introduction
Despite the lack of hard numbers on coaching results, the literature is filled with numerous articles on the types and processes of coaching (Cunningham, 1991; Diedrich, 1996; Flaherty, 1999; Hargrove, 1995; Goldsmith et al., 2000; Judge and Cowell, 1997; Kilburg, 2000; Koonce, 1994; Saporito, 1996; Thach and Heinselman, 2000) , and in the last few years many new organizations and consulting firms have sprung up to offer coaching as a service (Hall et al., 1999) . In anecdotal conversations with executives and human resource professionals, most profess to be pleased with the results, but are hard-pressed to provide quantitative data to support their viewpoint.
Introduction
This study attempts to fill part of that gap by describing the quantitative impact of a coaching and 360 feedback process on the leadership effectiveness of 281 executives. A short review of the coaching and 360 feedback literature is provided as background for the development of the process.
Introduction
Defining executive coaching Kilburg (1996) defines executive coaching as:
Introduction
A helping relationship formed between a client who has managerial authority . . . in an organization and a consultant who uses a wide variety of behavioral techniques and methods to help the client achieve a mutually identified set of goals . . . within a formally defined coaching agreement.
Introduction
He continues by stating that the goals generally are for the purpose of:
Introduction
Improving (the client's) professional performance and personal satisfaction and, consequently, to improve the effectiveness of the client's organization . . . . This definition is very inclusive of most concepts espoused in other descriptions provided in the literature (Flaherty, 1999; Hargrove, 1995; Goldsmith et al., 2000; Judge and Cowell, 1997; Koonce, 1994; Thach and Heinselman, 2000) .
Introduction
The literature clearly delineates between two major types of coaching (Koonce, 1994; Olivero et al., 1997; Snyder, 1995; Thach and Heinselman, 2000) . The first of these can be described as performance based, and focuses on practical and specific business issues. Examples might be goal setting, project management, or a specific interpersonal skill issue that is hindering the executive in achieving high-quality business performance. These types of issues might be a minor modification that a high potential manager needs to make, or be an issue that is causing potential derailment problems. This type of performance-based coaching is generally more short-term in nature, and can range from one to several meetings between coach and client.
Introduction
The second type of coaching can be defined as in-depth coaching. This can be more psychoanalytical in approach, attempting to get at deep-seated issues and often exploring personal values, motivations, and even family issues. This type of coaching generally lasts longer and is more costly. It is interesting to note, however, that there are Abstract Does executive coaching really work? Does it help improve leadershi p effectivenes s and productivity ? This action research study answers these question s by tracking the progress of 281 executive s participatin g in a six-month coaching and 360 feedback process. The results suggest that the combinatio n of multi-rater feedback and individua l coaching do increase leadershi p effectivenes s up to 60 per cent ± according to direct report and peer post-survey feedback . Implication s of the results for future executive development programs are discussed , and specifi c recommendation s are provided.
Introduction
some experts who recommend that executive coaching should not enter too far into the realm of psychology due to time restrictions and the fast-paced result-orientation of business (Judge and Cowell, 1997; Levinson, 1996; Synder, 1995) . Regardless of the type of coaching used, both are focused on helping the executive achieve goals which will enhance both organizational productivity and personal job satisfaction.
The coaching process
Many articles and books have been written on the process of``how to do coaching,'' and vary from three to six steps in length (Cunningham, 1991; Flaherty, 1999; Hargrove, 1995; Hudson, 1999; Goldsmith et al., 2000; Kiel et al., 1996; Koonce, 1994; Kilburg, 1996 Kilburg, , 2000 . Likewise, there are lists of coaching methods and techniques, which range from 360 assessment and feedback to training, simulations, role playing, and confrontations ± just to name a few. In general, however, the process of coaching includes the following three phrases: 1 contracting; 2 data collection; and 3 coaching.
The coaching process
In the contracting phase, the coach and client (which can sometimes be the individual client and/or the organization) develop an agreement regarding the coaching contract. This generally includes the goals of the coaching, estimated resources, time and confidentiality commitment, potential methods, and cost. Often several planning conversations occur in order to come to agreement on a written contract. The second phase involves some sort of data collection. This often takes the form of 360 assessments, personality and skill inventories, and/or one-on-one interviews by the coach with direct reports, peers, the manager, and customers of the client. The amount and length of the data collection phase is determined by the goals of the coaching relationship.
The coaching process
The largest amount of time is generally devoted to the third phase ± the coaching phase. Here the coach and client spend time analyzing the results of the data and designing a plan of action to overcome identified issues. Depending on how accepting the client is to the feedback, the time involved can vary in length. The coach may be required to use``confrontation techniques'' in order to help the client recognize and accept responsibility for the issues (Hall et al., 1999; Judge and Cowell, 1997; Thach and Heinselman, 1999) . The coaching phase also includes follow-up meetings to assess progress, observe client interactions; offer advice and support; and ± eventually ± to assist the client in evaluating the results of the coaching process and terminate the relationship.
The coaching process
Another impact on process is the type of coach used ± an internal or external coach. Hall et al. (1999) report that there are pros and cons to both types of coaches: externals are usually more costly, but are perceived by executives as being more objective; whereas internal coaches usually cost less and understand organizational culture, but are perceived as a confidentiality risk. The key to selecting the right coach and process appears to be a clear company goal around the use of coaches ± connected to an overall strategy with distinct expectations regarding results (Goldsmith et al., 2000; Olivero et al., 1997; Tanner, 1997) . Other success factors regarding executive coaching are linked closely to concepts identified in the best practices literature on executive development. These include linkage to business strategy, support and role-modeling by the top executive team, involvement of executives in design, implement as ongoing process, not program, and measure results (Bolt, 1999 (Bolt, , 1997 (Bolt, , 1985 O'Reilly, 1993; Vicere, 1996; Rothwell and Kazanas, 2000) .
The coaching process
The use of multi-rater (360) feedback with coaching According to Judge and Cowell (1997) , the best executive coaches use some form of 360 feedback with their client. Indeed, research has confirmed that the use of 360 feedback is one of the best methods to promote increased self-awareness of skill strengths and deficiencies in managers (Hagberg, 1996; Rosti and Shipper, 1998; Shipper and Dillard, 2000) . Multi-rater, or 360 feedback has been defined as the process of:
The coaching process
Gathering information about a person's behavior from a boss or bosses, direct reports, colleagues, . . . and other key stakeholders, such as customers and suppliers (Lepsinger and Lucia, 1997) .
The coaching process
Organizations that implement 360 feedback processes effectively generally link it to a specific business need, get top management buy-in, and provide ongoing support and follow-up (Waldman et al., 1998; Lepsinger and Lucia, 1997) . One of the most common follow-up methods is to use a personal coach. It is also recommended that when first implementing 360 feedback, that it be done for developmental rather than evaluative purposes (London and Smither, 1995) . A coach is an ideal support tool to use in a developmental situation.
The coaching process
Issues with 360 feedback are an absence of clear implementation purpose on the part of the organization, as well as empirical data on results (Waldman et al., 1998) . Another issue is the potential problem for abuse of the system by raters if they believe the data will be used for evaluative purposes, or if they become overwhelmed with the extra work 360 processes can generate and cynical of any real improvements. Because of this, careful planning should occur before implementing 360 processes in order that they be linked carefully to business needs and include appropriate communication and follow-up. The use of executive coaches to work with managers to decipher 360 results, as well as post survey methodology is recommended as an implementation option (Waldman et al., 1998; Goldsmith et al., 2000) Method Sample A total sample of 281 executives and high potential managers participated in this action research. They were all members of a mid-size, global, telecommunications firm with headquarters in the western USA. Specific criteria for the sample were set: must hold title of director, vice-president, or be identified as a high potential manager via succession planning data; must have worked in current position at the company for at least six months; and must volunteer to participate in the leadership development process ± not be coerced.
The coaching process
These criteria were developed as part of the establishment of a new executive development system within the company, which included this 360 feedback and coaching process.
Procedure
The procedure was designed based on best practice literature in executive development (Bolt, 1999; 1985; O'Reilly, 1993; Vicere, 1996; Rothwell and Kazanas, 2000) . This included gaining the support of the CEO and top executive team, and linking the 360 feedback process and coaching to a business need of developing more leaders for the future. Specifically, this entailed two goals: 1 increasing the percentage of overall leadership effectiveness on a post 360 survey; and 2 increasing the number of``ready now'' succession planning candidates from 1.0 to 2.0 leaders for the top 60 strategic positions in the firm.
Procedure
The executive development system was also designed to be linked to other human resource systems in the organization, such as performance management, recruiting, staffing, and succession planning. However, it was decided that for the first few years, the 360 feedback process would only be used for developmental purposes and not be linked to promotion and compensation. The 360 feedback and coaching process was designed to be the``heart'' of the executive development system, but it was supplemented by other development options, such as a strategic forum series; exchange dialogues with the CEO; shadow boards; targeted external education programs; and a series of on-the-job development options and community service. Support to participate in these other options was gained through the developmental plan created jointly by the executive and coach after completing the 360 feedback process.
Procedure
The procedure was implemented in three phases over a period of almost three years.
Phase one: develop and pilot 360 feedback process (eight months)
During phase one, the 360 feedback survey was co-designed by an external consultant and the top executive team, including the CEO. The consultant interviewed each member of the executive team to identify leadership competencies needed to achieve the organizational strategy for the next five years. The consulting firm then wrote the survey using an existing database of 360 competency questions which had a high rate of validity and reliability, however, they did include a few customized questions to evaluate new competencies identified by the executive team.
Phase one: develop and pilot 360 feedback process (eight months)
The final 360 survey included 76 items measuring 17 leadership competencies using a five-point satisfaction scale, with 1 being`v ery unsatisfied,'' and 5 being``very satisfied.'' It took approximately 15-20 minutes to complete and was first published in a paper format (later moved online to Web-based format). It was completed by peers, direct reports, manager, and self, and then mailed to an external survey processing firm to maintain confidentiality. Final results were mailed directly to the executive coach for review. The coach then met with the executive for one hour, as part of a one day training and survey debrief session. The one hour meeting allowed the coach to assist in the analysis of the 360 feedback and preliminary drafting of a development plan based on the results During phase one, the 360 survey was piloted on 57 executives, including the CEO and top executive team. These data were not included in the action research study of the 281 executives, because the process and some of the survey questions were modified as a result of pilot feedback. In order to obtain pilot feedback, each of the 57 executives was interviewed via phone or email questionnaire six months after the coaching debrief to assess their reaction to the 360 survey, coaching process, and discuss results of their development plan. Based on the pilot feedback, it was decided that more follow-up coaching sessions were necessary in order to keep the development plan``on the front burner,'' rather than pushed to the back due to the normal hectic schedule of a busy executive. In addition, it was decided that executives should only focus on one to three development actions and one strength, rather than a``laundry list'' of items which became overwhelming. Finally, some minor modifications were made to the 360 survey items in order to enhance face validity.
Phase two: implement year one 360 feedback with coaching (12 months)
Phase two was launched with 168 executives over a one year period. The process was originally only budgeted corporately for 100 executives, but it became so popular that many executives offered to fund it through their department budgets. In this phase, the modified 360 survey was used, and executives not only received a one-hour coaching debrief session during the one-day training class, but also had three additional follow-up sessions with their coaches over a period of six months, before a mini 360 and email post survey was administered. The coaching sessions were scheduled as follows: 1 First coaching session. A one-hour debrief of the 360 survey during the training class. This included analysis of results, discussion, and preliminary preparation of a development plan. The coach also helped the executive narrow down the list of one to three development actions and one strength. During the training class, the executive was instructed on how to follow-up with their manager, peers, and direct reports to thank them for the 360 feedback and seek their assistance in identifying specific actions to resolve the one to three identified development items. The executive was also instructed to show their draft development plan to their manager, in order that the manager could provide input to the plan and provide approval. However, the actual 360 data was not shown to the executive's manager, but was kept confidential between the executive and coach. A one hour face-to-face or phone coaching conversation in which the coach and executive repeated the same steps as outlined in the above coaching session, plus prepared for the mini 360 post and email surveys. 5 Mini 360 and email post surveys administered at six months. At six months, the external survey processing firm mailed mini 360 post surveys to the executive to distribute to peers, direct reports, and managers again. The mini 360 survey was composed only of the one to three items the executive had selected for development, as well as three standard questions which were used as the basis of this action research. These questions focused on percentage increase in leadership effectiveness and follow-up frequency. In addition, an email post survey was sent from the corporate executive development office to each executive to seek feedback on the number of times the executive met with the coach, self-assessed progress, and pros/cons of the process.
Phase two: implement year one 360 feedback with coaching (12 months)
Although the procedure called for the coach and executive to meet for a total of four times, there were some cases where the coach and executive had many more contacts via voicemail and email ± especially to schedule follow-up meetings, given the very crowded schedules of most executives. Also funding was made available through the corporate executive development office for four coaching sessions, however executives had the option of contracting with their coach for additional sessions if they paid for it from their own budget.
Phase three: implement year two 360 feedback with coaching (12 months)
Phase three was launched with 113 executives during the third year. Again, the corporate budget funded 100 executives for the year, but due to budget cuts across the company during the third year, fewer executives elected to fund the process from their own department budgets. Phase three followed the identical procedure outlined in phase two, with the exception of some minor modifications to the one-day training session, which had received some process improvement suggestions via the post survey feedback from the previous year. Also, the coaches were urged to counsel the executives to complete the mini 360 post survey. This had not been very well emphasized in phase two, resulting in a lower percentage response rate.
Analysis
The data resulting from the mini 360 and email post surveys were analyzed in two ways. The individual mini 360 post survey data were analyzed by the external survey firm using descriptive statistics on the six items and mailed back in report format to the executive and coach. Then a compilation of all the data was prepared for the company, using the first three items. In addition, they correlated the impact of follow-up frequency on leadership effectiveness. The data resulting from the email post survey were compiled by the corporate executive development office using descriptive statistics for the first three items. A qualitative methodology of thematic coding and frequency was used to analyze the last two items regarding what worked well and areas for improvement regarding the overall process.
Results
Results for this action research study are provided in three areas: 1 participation rates; 2 leadership effectiveness; and 3 coaching impact.
Results
Each of these are described in the following sections.
Participation rates
Overall, participation rates were higher than expected, with more executives volunteering to complete the process than the corporate budget would merit. Positive reaction was attributed to the role-modeling and encouragement of the CEO and executive team to complete the process. Drop out rates were low (13 per cent in phase two and 7 per cent in phase three), and had to do with transfers within the company or resignations. Participation in completing the mini 360 and email post surveys was not mandatory, and therefore the overall response rates ± 47 per cent and 45 per cent respectively ± were considered good. A total of 537 raters, including direct reports, peers, and the manager of the executive, participated in the mini 360 post survey. Table I illustrates the results from the mini 360 post survey. It shows that the overall impact on leadership effectiveness, due to the six month coaching and 360 survey process, was an average increase of 55 per cent with the phase two executives and 60 per cent with the phase three executives ± as perceived by others. Self leadership effectiveness ratings at the end of the six months coaching and 360 feedback process were slightly lower at 52 per cent and 56 per cent, respectively. From Table I it is obvious that fewer peers responded to the mini 360 post survey than direct reports. The reason for this is because the executives were instructed to send the post survey to only those raters who were directly impacted by the one to three development items upon which they were working. In many cases, the development items addressed interactions with direct reports, rather than peers. Likewise, the number of overall raters on the mini 360 post survey is smaller than the number who responded to the original 76 item 360 survey, because ± again ± the executive was focusing on rater's perceptions on only the one to three development items. Rater response requirements were that at least three raters had to respond to an item, before it could be compiled and reported.
Leadership effectiveness
The data were also analyzed to examine frequency of follow-up, as well as the impact of follow-up on per cent increase in leadership effectiveness. Executives were urged to follow-up with impacted raters ± either direct reports and/or peers ± on three occasions, as well as to keep their manager informed of progress. The first occasion was discussing the 360 feedback with raters and thanking them for participating within at least 30 days of receiving the feedback report.
Leadership effectiveness
Executives were then counseled to follow-up informally with raters to assess progress on the one to three identified development areas on two more occasions ± generally every 60 days, before the mini 360 post survey was distributed at six months. Table II and Figure 1 illustrate these results.
Leadership effectiveness
The data illustrate that there is a positive correlation to follow-up frequency and per cent increase of leadership effectiveness. However there is a disturbing trend of outliers in both phase two (n = 2) and phase three (n = 4) where per cent increase in leadership effectiveness is rated at 65 per cent and 60 per cent respectively with no follow-up.
Coaching impact
The impact of coaching was primarily measured on the email post survey in which executives were asked to report the number of times they actually met with their coach; whether or not they completed their development plan, and their self-reported increase in leadership effectiveness. Interestingly enough, executives reported lower self-assessment on per cent increase in leadership effectiveness on the email post survey (36 per cent average) than they did on the mini 360 post survey (53 per cent). The average number of times they met with their coach was 3.6, versus the actual four times outlined in the process. This was consistent for both phase two and three participants and speaks to the hectic schedule of executives. Finally, of the 114 executives responding to this post survey, 108 (95 per cent) reported that they completed implementing their development plan.
Coaching impact
With this survey, it was also possible to correlate the number of times an executive met with their coach to self-reported per cent increase in leadership effectiveness. Figure 2 illustrates these results below.
Coaching impact
Since only 17 executives exceeded four coaching sessions, it is difficult to claim a positive correlation in this analysis, however it is possible to state that completing three to five coaching sessions appears to have a much more positive impact on self-reported per cent increase in leadership effectiveness than only one or two coaching sessions.
Thematic analysis ± what worked well?
The thematic analysis of the qualitative data provided much more rich detail on the benefit of the coaching process. In the descriptive section regarding``what worked well,'' a total of 125 comments were coded into seven major themes, utilizing a decision rule of at least two coded items to create a theme. Table III illustrates these themes.
Thematic analysis ± what worked well?
The major theme that emerged as being the most positive part of the process was the coach at 43 items, or 34 per cent of the responses. Examples of some of the comments include:
Thematic analysis ± what worked well?
It is great having a coach with an outside perspective giving candid feedback.
Thematic analysis ± what worked well?
[The coach's] honest, direct input has been very valuable over the past months.
Thematic analysis ± what worked well?
It is helpful to have someone to talk to that is independent and provides another point of view.
Thematic analysis ± what worked well?
[My] coach is knowledgeable, professional, and helpful in providing support and direction.
Thematic analysis ± what worked well?
The one-on-one with the coaches helped me the most. The second highest scoring category was the 360 feedback. Here respondents reported that receiving the feedback was very useful. Some comments include:
Thematic analysis ± what worked well?
I like the feedback process from my peers and direct reports, and that this portion of it is confidential so that they can be honest and direct.
Thematic analysis ± what worked well?
I appreciated receiving the feedback, particularly from my Direct Reports.
Thematic analysis ± what worked well?
Direct feedback provides the best basis for personal growth.
Thematic analysis ± what worked well?
Other useful categories included the overall process and the focus on a few specific development items. Another item that supported the benefit of the coaching was a category coded as goad, in which executives reported they appreciated the fact that the coach was there to``goad'' them into implementing their development plan and follow-up with raters. As one participant stated:
Thematic analysis ± what worked well?
[You are] compelled to complete [the process] ± as you knew your coach would be following-up.
Thematic analysis ± areas for improvement
The qualitative comments regarding improvement are tabulated in Table IV . Here eight themes appeared, but only totaling 49 comments, compared to the 125 comments volunteered in the``what works category.'' In this section, the major theme was none ± or no areas for improvement, with the second area being coach match and the third theme as organizational changes. This third category had to do with changes in staff, new assignments, or large organizational change issues that interfered with completing the six month process. Regarding the coach match theme, some of the executives had a``match issue'' with their coach:
Thematic analysis ± areas for improvement
My coach seemed too affirming and may not have been candid enough.
Thematic analysis ± areas for improvement
I'm not sure how effective the coach is. It may depend on the particular coach.
Thematic analysis ± areas for improvement
Some of my staff felt they didn't have a good match with a coach, therefore value was not as high for them.
Thematic analysis ± areas for improvement
The overall process attempted to deal with these issues up front during the first training A final category that reflected positively on the impact of the coach had to do with the code more coach, in which executives expressed the need to have more coaching sessions than just the four sessions funded under the corporate executive development process.
Discussion and conclusions
This action research demonstrates the positive effect executive coaching and 360 feedback have on increasing leadership effectiveness. On the mini 360 post survey, direct reports, peers, and managers rated leadership effectiveness as having increased an average of 55 per cent and 60 per cent respectively during the two phases, as a result of the overall coaching and 360 feedback process. Furthermore, this study illustrates that more frequent follow-up and interaction with 360 raters during the coaching and improvement cycle, results in higher ratings of leadership effectiveness on the post 360 survey. Finally, this study supports the positive impact of coaching in terms of developing leaders. The themes identified in the qualitative data illustrate this, and the coaching correlation graph suggests that their may be a connection to the number of coaching sessions and self perceived leadership effectiveness.
Discussion and conclusions
Regarding the achievement of the identified goals for the corporation, both were accomplished. As verified by the post surveys, leadership effectiveness did improve among executives as perceived by stakeholders, and also the number of``ready now'' candidates was increased from 1.0 to 2.0 at the end of the first two years.
Limitations of the study
The positive results demonstrated in this study are hampered, however, by its action research design. No effort was made to isolate variables, since a``complete process'' was being implemented. Therefore it is difficult to separate the impact of the 360 feedback from the executive coaching. One wonders what the results would have been if executives had only received their 360 feedback report and no coaching. This question was partially answered at the end of phase one when the 57 executives in the pilot study suggested that more coaching be included, but no quantitative baseline was established.
Limitations of the study
Another issue had to do with the follow-up process. Here the step of following-up at least three times over six months with direct reports, peers, and managers to discuss the feedback and assess progress on the identified action items appears to have had a very positive impact on rater's perceptions of leadership effectiveness. Obviously a few executives chose to skip this step or do less follow-up, and this had a somewhat negative effect on leadership effectiveness ± but not in all cases, since there were a few outliers. Why is this? What would happen if the follow-up step were eliminated from the complete process?
Limitations of the study
Another handicap was thè`m odifications'' made to the process over the two-year and eight-month period. Obviously in a business setting this type of process improvement is normal, but it does make it more difficult to report on action research. It is for this reason that the data had to be differentiated into phases, however, in the final analysis, there was not much difference in the overall results between participants in phase two verses phase three. A final limitation of this study is that it was conducted in only one organization. Implications and recommendations for executive development
Limitations of the study
Despite the drawbacks of action research, and the lack of``pure study design,'' it is rewarding to observe this type of progress on leadership development within a business setting. These positive results suggest that this type of simple, yet effective executive coaching and 360 feedback process should be implemented in other settings. Obviously the process would need to be adjusted to fit different organizational cultures and goals, but it appears that the combination of the three elements emphasized in this study ± 360 feedback; coaching; and follow-up with stakeholders ± are a winning combination. It may be useful for organizations to experiment with the difference between using internal versus external coaches in order to preserve costs. However, this should be done with respect towards maintaining confidentiality and quality of coaching. Organizations could also experiment with the number of coaching sessions, as well as the length of time between the pre and post 360. Finally, there is the question of how often executives should participate in this type of process ± once a year or more often. Should one consider implementing a philosophy of continuous improvement of leadership skills through this type of ongoing process?
Conclusion
In conclusion, this study supports previous findings on the positive impact of 360 feedback and executive coaching (Shipper and Dillard, 2000; Olivero et al., 1997; Hagberg, 1996) . It also reinforces themes identified in the best practice literature on executive development regarding the need to involve the top executive team in designing and role-modeling the process, as well as linking it to strategic organizational goals (Bolt, 1997) . By implementing these types of leadership development processes which emphasize 360 feedback to promote selfawareness combined with the support of a talented coach, leaders can be developed; high potentials managers can continue to hone their skills; and derailing executives can be saved.

{'paper_id': '10090842', 'title': 'The impact of executive coaching and 360 feedback on leadership effectiveness', 's2_url': 'https://api.semanticscholar.org/CorpusID:10090842', 'source': 'S2ORC'}