Original Investigation
DOI: 10.30827/Digibug.80313


Accuracy of subjective stats of key performance indicators in tennis


Precisión de las estadísticas subjetivas de indicadores clave del rendimiento en tenis


International Journal of Racket Sports Science, vol. 4(2) (July-Dec, 2022), Pag. 32-39 . eISSN: 2695-4508


Received: 29-11-21
Acepted: 02-12-22

AUTHORS

Takashi Jindo ORCID

Daisuke Mitsuhashi

Faculty of Health and Sport Sciences, University of Tsukuba

Tatsumasa Kubota

School of Pharmaceutical Sciences, University of Shizuoka


Corresponding Author: Takashi Jindo, Ph.D., jindo.takashi.ge@u.tsukuba.ac.jp

Cite this article as: Jindo, T., Mitsuhashi, D., & Kubota, T. (2022). Accuracy of subjective stats of key performance indicators in tennis. International Journal of Racket Sports Science, 4(2), 32-39.



ABSTRACT

Abstract

The compilation of stats by performance analysis is common in matches with top professional tennis players. However, outside the top level such objectively evaluated stats and feedback for players are rare. With this in mind, an original method was developed that asks players to subjectively evaluate the match stats. This study aimed to investigate the accuracy of subjective stats in tennis. The participants were 30 male collegiate athletes, including some who had participated in national-level competitions. The participants played a 6-game, 1-set practice match, and immediately after the match subjectively evaluated the stats of key performance indicators such as percentages, number of shots, and rally patterns. Objective stats were aggregated using video clips recorded by a digital camera or smartphone. Based on Bland-Altman plots show that subjectively evaluating their own performance indicators helped to confirm the objective stats. Although some variables showed fixed or proportional biases, the mean differences were not significant (percentage of first serve in: 1.733% points; double faults: 0.400 times; net plays: -0.767 times; unforced errors: -2.133 times). These findings support the implementation of a subjective evaluation of key performance indicators in tennis players who might have difficulty incorporating objective evaluations.

Keywords: Performance analysis, profiling, feedback, tactics.

Resumen

La recopilación de estadísticas mediante el análisis del rendimiento es común en partidos con jugadores profesionales de élite de tenis. Sin embargo, este tipo de estadísticas y retroalimentación evaluadas objetivamente son poco frecuentes en los niveles de rendimiento inferiores. Teniendo esto en cuenta, se desarrolló un método original que pide a los jugadores que evalúen subjetivamente las estadísticas de juego. El objetivo de este estudio era investigar la precisión de las estadísticas subjetivas en tenis. Los participantes fueron 30 atletas hombres universitarios; algunos de ellos habían participado en competencias nacionales. Los participantes jugaron un partido de práctica a 6 juegos y 1 set, e inmediatamente después evaluaron subjetivamente las estadísticas de indicadores clave del rendimiento tales como porcentajes, número de golpes y patrones de intercambio de golpes. Se añadieron estadísticas objetivas a través de videos grabados con una cámara digital o un teléfono inteligente. Los gráficos de Bland-Altman sugieren que evaluar subjetivamente sus indicadores de rendimiento les ayudó a confirmar las estadísticas objetivas. Aunque algunas variables mostraron sesgos fijos o proporcionales, las diferencias medias no fueron significativas (porcentaje de primeros saques: 1,733 % puntos; dobles faltas: 0,400 veces; jugadas de red: -0,767 veces; errores no forzados: -2,133 veces). Estos hallazgos apoyan la implementación de una evaluación subjetiva de los indicadores clave del rendimiento en jugadores de tenis con dificultades para aplicar evaluaciones objetivas.

Palabras clave: Análisis del rendimiento, perfilación, retroalimentación, tácticas.



Introduction

INTRODUCTION


Various sports have adopted performance profiling to analyze and improve athletic performance (Butterworth et al., 2013). A traditional method introduced by Butler and Hardy (1992) set key performance indicators (KPIs) between a coach and player prior to the match, and then rated the quality of performance on a zero to ten scale. A previous study reported that the majority of coaches praised the usefulness of the performance profiles as part of the wider coaching process (Butterworth et al., 2012). In addition, other studies have reported that athletes believed producing performance profiles in a group setting to be generally very useful (Weston et al., 2011a), thus performance profiling might enhance intrinsic motivation (Weston et al., 2011b).

Meanwhile, even for critical events in a game, coaches’ observational accuracy is not perfect (Franks & Miller, 1986; Laird & Waters, 2008), leading to objective performance analysis and performance profiling using video clips or specific software being adopted recently (Butterworth et al., 2013; O’Donoghue, 2005). In these methods, the number of executions, successes, or errors of each performance indicator is objectively aggregated (O’Donoghue, 2005). Other than for world top level athletes, however, in most cases neither coaches nor tournament hosts conduct objective performance analysis. One study reported that the proportion of coaches who engage in performance analysis using video footage decreases corresponding to the lower athletic level of the player (Kraak et al., 2018). In tennis, even at a top level professional tournament, detailed objective stats would not necessarily be provided to the players (Kovalchik, 2021), while lower-level tournaments probably do not record the stats. Therefore, while performance analysis with objective stats confirms the accuracy, barriers for conducting it at various athletic levels may exist.

Compiling the values of performance indicators would be effective for improving performance. On the other hand, athlete-centered or continuous effort has been reported as an important aspect of performance profiling (Weston et al., 2011b). Traditional performance profiling has been used as a subjective tool whereby athletes would assess themselves against one or more KPIs (Butterworth et al., 2013). However, the accuracy of subjective statistics has not been well investigated. Mitsuhashi (2002) reported that subjective stats had underestimated own first service (17.1 %), and overestimated the total number of opponent forehand stroke winners (twice) and backhand stroke errors (4.4 times). However, because the study included only ten participants in five matches, accumulated evidence is needed to understand the accuracy of subjective stats.

A previous study reported that tactical skills were greater in higher level tennis players (Kolman et al., 2019). One likely reason for this is that higher athletic level players make decisions regarding tactics based on accurate subjective evaluation of their performance as well as their opponents’ performance during a match. On the other hand, although performance profiling is usually conducted for a specific performance, analyzing that of an opponent is important for tactical planning, especially in ball games. It is anticipated that accurate subjective evaluation might be effective for improving tactics or performance through repetitive feedback and feedforward not only for one match but over a full tournament or season. Based on these assumptions, this study also focused on the difference in the accuracy of subjective stats between athletic levels and accuracy of opponent performance evaluation.

The purpose of this study was to investigate the accuracy of subjective KPI stats in tennis, which will be useful for creating a performance profile with a new method. Moreover, the accuracy of subjective stats for the opponent’s performance indicators and the differences between athletic levels were also investigated.


Methods

MATERIAL AND METHODS


Participants

This was an observational study targeting singles tennis practice matches. The participants were 30 male collegiate tennis players at a university affiliated with the Kanto Inter-Collegiate Tennis Federation in Japan. They engaged in hitting practice and training for approximately five days a week, three to four hours per session. The participants were recruited using convenience sampling. Data gathering was conducted as part of extracurricular activities. Of the total participants, 19 had taken part in a national or equivalent-level junior or collegiate tournament (higher achievement group; age range: 18-21 years) while the remaining 11 participants had lower achievement (lower achievement group; age range: 18-21 years). The competitive records of the higher achievement group were as follows: two had participated in national-level collegiate tournaments, three had participated in semi-national-level collegiate tournaments, and 14 had participated in national-level junior tournaments. Objective competitive record was not available for the participants of lower achievement group; however, they had participated in the same hitting practice with the higher achievement group and training and considered to be regional competitive level.

The ethical committee of the author’s affiliation approved the study protocol (approval number: TAI021-113).

Design and procedure

The practice matches were played by players who were considered to be equivalent or at a close athletic level based on each participant’s previous achievement. For the participants who had equivalent record or those who did not have available record, practical experience or knowledge were adopted to decide the match combination. Accordingly, nine matches among the higher achievement group, one match between high and lower achievement groups, and five matches among lower achievement groups were played. The scoring format used in this study was a 6-game, 1-set match, with advantages and tie-break scores. Prior to the match, participants were notified that subjective and objective evaluations of their indicators and their opponents’ indicators would be conducted. Training or familiarization session for subjective evaluation was not conducted.

The evaluation of subjective stats was conducted by each player immediately after the match using an original score sheet developed by the authors (Figure 1). Using the sheet, an original method was conducted that asked players to subjectively observe and record the values for the KPIs. Specifically, the players were asked players to recall and fill out the percentages of success or error of each shot or rally indicators on a score sheet immediately after a match. If this subjective evaluation has a certain accuracy, a performance profile can be created. This method might easily and continuously be applied by players who have difficulty implementing objective evaluations such as athletes at the lower levels. Objective stats were aggregated using video clips recorded using a digital camera or smartphone.

v4n2a4image001.png

Figure 1. Score sheet

Performance indicators of subjective and objective stats

This study focused on the indicators of serve, return, and rally, which are commonly used in the performance analysis of tennis (O’Donoghue, 2005). Percentages, numbers of own and opponent shots, and rally indicators, were recorded.

For service, percentages of first service in, percentage of points won when first service in, and total number of double faults were used. For the return, the percentage of return in was used. For rally, the total number of winners, unforced errors, net plays, and percentage of points with net plays were used.

Analyses

The mean and standard deviation (SD) of each indicator were calculated. The Shapiro-Wilk test was used to check the normal distribution of the variables. We created a Bland-Altman plot (Bland & Altman, 1986), and conducted a paired t-test or Wilcoxon signed rank test between subjective and objective stats to investigate the accuracy of the former. The differences in the stats between the two methods were calculated by subtracting the objective values from the subjective values. Accordingly, more positive values indicate subjective overestimation and more negative values indicate subjectively underestimated stats. In a Bland-Altman plot, agreement and normality of distribution (i.e., whether the values lie between ±1.96SD) between the two stats was confirmed (Bland & Altman, 1986). Fixed bias was examined using a one-sample t-test that calculates the mean differences of two ways that significantly differ from zero. Proportional bias was examined using either the Pearson or Spearman correlation coefficients. Bland-Altman plots were shown only for their own performance indicators, which was the main target of this study. In addition, a two-sample t-test was performed to detect differences according to athletic achievement levels.

The analysis was conducted using R version 4.1.1 (The R Foundation). The level of statistical significance was set at P < 0.05.


Results

RESULTS


Performance indicators of own shots and rally

For own shot and rally indicators, significant differences between subjective and objective stats were observed in the total number of unforced errors (P = 0.011) and net plays (P = 0.008) (Table 1). Although most cases lay between ±1.96SD on the Bland-Altman plots, some variables had one or two cases less than -1.96SD (Figure 2). For fixed bias, overestimation was observed in the total number of double faults (P = 0.020), while underestimation was observed in the total number of unforced errors (P = 0.009) and net plays (P = 0.006). The other variables showed no fixed bias. For proportional bias, a tendency for overestimation in the higher percentages in first serve (r = 0.51, P = 0.004) and underestimation in fewer net plays (ρ = -0.39, P = 0.033) were observed. The other variables showed no proportional bias.

Performance indicators of opponent shots and rally

For opponent shot and rally indicators, significant differences between subjective and objective stats were observed in the percentages for first serve (P < 0.001) and points won on first serve (P = 0.024), total number of winners (P = 0.018), unforced errors (P = 0.015), and net plays (P = 0.023). For fixed bias, overestimation was observed in the percentages in first serve (P < 0.001) and points won when the first serve was in (= 0.026), while underestimation was observed in the total number of winners (P = 0.015) and unforced errors (P = 0.011). The other variables showed no fixed bias. For proportional bias, a tendency of underestimation in the fewer number of winners (ρ = -0.55, P = 0.001) was observed. The other variables showed no proportional bias.

Table 1. Differences in subjective and objective stats between all participants

Item Subjective stats Objective stats Mean difference between subjective and objective stats P-value
mean ± SD mean ± SD
Own shot and rally indicators
Percentages of 1st service in, % 56.5 ± 16.3 58.2 ± 11.1 -1.733 0.386
Percentage of points won when 1st service in, % 60.7 ± 11.7 61.9 ± 10.2 -1.200 0.461
Total number of double faults, times 2.1 ± 1.6 1.7 ± 1.3 0.400 0.022
Percentage of return in, % 67.9 ± 8.4 68.5 ± 10.3 -0.600 0.772
Total number of winners, times 5.2 ± 3.1 6.0 ± 3.4 -0.800 0.227
Total number of unforced errors, times 17.1 ± 5.1 19.2 ± 5.4 -2.133 0.011
Total number of net plays, times 2.8 ± 3.6 3.6 ± 4.4 -0.767 0.008
Percentages of points with net play, % 53.2 ± 35.8 58.7 ± 30.5 -5.467 0.590
Opponent shot and rally indicators
Percentages of 1st service in, % 66.9 ± 8.9 58.2 ± 11.1 8.700 <0.001
Percentage of points won when 1st service in, % 67.3 ± 10.1 61.9 ± 10.2 5.467 0.024
Total number of double faults, times 1.8 ± 1.7 1.7 ± 1.3 0.167 0.403
Percentage of return in, % 70.3 ± 8.0 68.5 ± 10.3 1.800 0.733
Total number of winners, times 4.8 ± 2.1 6.0 ± 3.4 -1.133 0.018
Total number of unforced errors, times 16.3 ± 4.1 19.2 ± 5.4 -2.867 0.015
Total number of net plays, times 3.0 ± 5.3 3.6 ± 4.4 -0.567 0.023
Percentages of points with net play, % 51.9 ± 36.5 58.7 ± 30.5 -6.800 0.592

Bold numbers indicate P < 0.05. The dagger (†) indicates that any of the analyzed variable were not normally distributed, and Wilcoxon signed rank test was used.

SD: standard deviation

v4n2a4image002.png

Figure 2. Bland-Altman plot of method comparison between subjective and objective stats (continued)

Comparison between athletic achievement level

A significant difference was observed in the percentages in first serve between the low and high achievement groups (P = 0.018). Specifically, the higher achievement group (-4.9±11.2 percentage points) subjectively underestimated the stats compared to the lower achievement group (3.7±7.1 percentage points). The other variables showed no significant differences between groups.

Table 2. Comparison of the subjective and objective stats difference between athletic achievement level

Item Low achievement group (n=11) Low achievement group (n=19) P-value
mean ± SD mean ± SD
Own shot and rally indicators
Percentages of 1st service in, % 3.7 ± 7.1 -4.9 ± 11.2 0.018
Percentage of points won when 1st service in, % -2.3 ± 10.6 -0.6 ± 7.5 0.655
Total number of double faults, times 0.6 ± 0.8 0.3 ± 0.9 0.262
Percentage of return in, % 0.5 ± 14.0 -1.2 ± 9.2 0.735
Total number of winners, times -0.8 ± 1.6 -0.8 ± 3.4 0.975
Total number of unforced errors, times -1.3 ± 4.6 -2.6 ± 3.8 0.428
Total number of net plays, times -0.6 ± 1.3 -0.8 ± 1.5 0.698
Percentages of points with net play, % 5.3 ± 33.1 -11.7 ± 34.5 0.206
Opponent shot and rally indicators
Percentages of 1st service in, % 8.6 ± 12.0 8.7 ± 8.7 0.981
Percentage of points won when 1st service in, % 5.4 ± 13.0 5.5 ± 12.7 0.974
Total number of double faults, times -0.1 ± 0.8 0.3 ± 1.1 0.265
Percentage of return in, % 4.8 ± 7.2 0.1 ± 11.9 0.192
Total number of winners, times -0.5 ± 1.7 -1.5 ± 2.6 0.195
Total number of unforced errors, times -0.7 ± 5.2 -4.1 ± 5.8 0.120
Total number of net plays, times -0.8 ± 1.4 -0.4 ± 1.8 0.508
Percentages of points with net play % -7.5 ± 28.3 -6.4 ± 39.8 0.936

Bold numbers indicate P < 0.05

SD: standard deviation


DISCUSSION

DISCUSSION


This study investigated the accuracy of subjective stats of own and opponents’ KPIs in targeted tennis practice matches of collegiate male players. The results confirmed the accuracy of subjective stats for their own KPIs. On the other hand, half of the opponent’s KPIs showed fixed or proportional biases between subjective and objective stats, which indicates less accuracy in opponent performance evaluation compared to their own. There was no significant difference between athletic achievement levels, except for the percentage of own first serve in, whereby we did not confirm the accuracy difference according to athletic level. These findings may be useful when conducting subjective evaluations and subsequent performance profiling.

In this study, the Bland-Altman plot showed a high degree of agreement, and there were no significant differences or fixed or proportional biases between the subjective and objective stats of most variables of own performance indicators. Even for the variables that showed fixed or proportional biases, the mean differences were not significant (percentage of first serve: -1.733 percentage points; double faults: 0.400 times; net plays: -0.767 times; unforced errors: -2.133 times), so this can be considered an acceptable level in a practical setting. A previous study that examined the accuracy of subjective stats had a small number of participants (Mitsuhashi, 2002) and there is a dearth in knowledge about the accuracy of subjective evaluation of tennis. This study suggests that performance profiling based on subjective statistics could create a profile with a certain accuracy. Such a method can be implemented with an athlete-centered and continuous style, which is expected to have consequences such as increasing the intrinsic motivation of athletes (Weston et al., 2011b). In the case of applying this method, it should be considered that, depending on the items and their levels, biases may occur.

In terms of the percentage of first serve in, where proportional bias was observed, the higher the probability, the more overestimation was observed. It would appear that when the percentage is low, players evaluate their own performance more negatively and, conversely, when the percentage is high, they evaluate it more positively. Such bias may affect the tactical and psychological aspects of a match. The number of net plays was also found to have a proportional bias, but this result may be highly influenced by one player who plays extremely frequent net plays. For a frequent net player, not only the number of net play attempts but also detailed information such as the method of approach and the characteristics of the opponent's pass might be important, and it may be necessary to consider how to utilize the subjective evaluation specifically for such a player.

On the other hand, compared to objective, the subjective stats of the opponent's performance indicators showed significant differences or biases in the percentages and points of first serve in and total numbers of winners, unforced errors, and net plays. Moreover, the tendency of overestimation and underestimation differed for each item. Although not much different from the objective stats, less accuracy was observed compared to the performance indicators. Although the practice matches in this study instructed the player to remember the stats, some information was not accurately recalled. In this context, these biases may be due to the fact that in planning tactics during a match, players must not only evaluate KPIs but also perceive a variety of other information such as the opponent's type of shot, ball speed, and course. Future studies should clarify what aspects of the opponent's performance each player focuses on during a match will lead to proposals for effective subjective evaluation after the match.

With regard to athletic achievement levels, a significant difference was found only in the own first serve probability, where the high achievement group tended to estimate their own performance lower. It should be noted that the percentage of first serve in this study was comparable to the average stats of junior and professional players (around 60%) (Kovalchik & Reid, 2017). In this study, it was hypothesized that the higher the level of athletic achievement, the more accurate the subjective evaluation, due to its importance for tactical decision-making. The hypothesis, however, was not supported, as no differences were found except for the percentage of own first serve in. In other words, the results showed that differences in tactical skills by competition level (Kolman et al., 2019) were not based on an accurate subjective evaluation of KPIs. Further investigation is needed to determine the role of accurate subjective evaluations in matching performance.

As a practical implication, it is possible to create a profile and grasp fluctuations in performance by using the score sheet to continuously record one’s own performance stats. In addition, whether repeating such subjective evaluations changes the accuracy of subjective stats and whether it improves performance and affects intrinsic motivation should be examined. Moreover, we focused on the KPIs of own and opponent performance, but items should be selected according to the performance of each player. Since the KPIs might be stable in a certain value (i.e., the percentages for first serve usually fell into around 50 to 60 % (Kovalchik & Reid, 2017), more detailed subjective evaluation such as service or return stats in deuce- and advantage-side, shots’ courses or values that depend on situations (e.g., beginning or later of the match, game, etc.) should be adopted. In addition, not only shots performance but also movement performance such as movement speed or distance (Reid et al., 2016) would be a candidate indicator. Moreover, different indicators for own and opponents’ performance for each player might be beneficial because limited resources can be focused on an important aspect.

This study has several limitations. First, with regard to the setting of the game, we informed the participants in advance of the matches that we would conduct a subjective evaluation, which may have impacted the accuracy of the results. In addition, because the matches were played with one set match, which is fewer than in most official matches (three sets), this may have affected the results. Future studies should target actual matches, and consider the length of the matches and differences in the match pattern. At the same time, the effect of familiarization on subjective evaluation, which was not considered in this study, should be accounted for. Second, the insufficient participants’ information and the participants’ selection might also be limitations. Based on international standards of competition level provided by the International Tennis Federation (ITF), the high group corresponded with intermediate to advanced, and the low group with novice to intermediate. Other characteristics, such as competitive experience (i.e., age, number of participated tournament), styles of play, or training and practice condition might affect the subjective evaluation. Future studies should investigate this information and examine whether the results of this study can be replicated at other levels and participants should be investigated. Finally, the roughness of the evaluated KPIs reveals a limitation for the application of the findings. Specifically, the accuracy of detailed indicators such as service or return stats in deuce- and advantage-sides, and shots’ courses or values that depend on situations (e.g., beginning or later of the match, game, etc.) are unknown. Investigating their accuracy could demonstrate whether these detailed stats evaluations would be useful in a practical setting.


Concluding

CONCLUSIONS


This study confirmed that the subjective stats of own performance indicators in tennis has a certain accuracy. This suggests that conducting performance profiling based on subjective statistics could be useful. On the other hand, the subjective stats of the opponent's performance indicators were not as accurate as that of their own performance indicators. This might be because the players have focused on not only their own KPIs but also on the other opponents’ performance indicators during a match. In addition, there was no significant difference in athletic achievement levels. Based on these findings, understanding the perspectives of analysis of one’s own and opponents' performance by players at various athletic levels will lead to proposals for effective ways of reflecting on matches.

Acknowledgments

We would like to express our gratitude to all the participants and Mr. Daiki Hirota for their cooperation in this study.

DECLARATION OF CONFLICT OF INTEREST

The authors declare that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper.


References

REFERENCES


Bland, J. M., & Altman, D. G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. Lancet, 1(8476), 307-310.

Butterworth, A., O’Donoghue, P., & Cropley, B. (2013). Performance profiling in sports coaching: a review. International Journal of Performance Analysis in Sport, 13(3), 572-593. https://doi.org/10.1080/24748668.2013.11868672

Butterworth, D. A., Turner, J. D., & Johnstone, A. J. (2012). Coaches’ perceptions of the potential use of performance analysis in badminton. International Journal of Performance Analysis in Sport, 12(2), 452-467.

Franks, I. M., & Miller, G. (1986). Eyewitness testimony in sport. Journal of Sport Behavior, 9(1), 38-45.

Hardy, B. R. J., Butler, R. J., & Hardy . (1992). The performance profile: theory and application. The Sport psychologist, 6(3), 253-264. https://doi.org/10.1123/tsp.6.3.253

International Tennis Federation. ITN Categories. Retrieved 27 October from http://www.tennisplayandstay.com/itn/itn-categories/itn-categories.aspx

Kolman, N. S., Kramer, T., Elferink-Gemser, M. T., Huijgen, B. C., & Visscher, C. (2019). Technical and tactical skills related to performance levels in tennis: A systematic review. Journal of Sports Sciences, 37(1), 108-121.

Kovalchik, S. (2021). Why Tennis Is Still Not Ready to Play Moneyball. Harvard Data Science Review.

Kovalchik, S. A., & Reid, M. (2017). Comparing matchplay characteristics and physical demands of junior and professional tennis athletes in the era of big data. Journal of sports science & medicine, 16(4), 489.

Kraak, W., Magwa, Z., & Terblanche, E. (2018). Analysis of South African semi-elite rugby head coaches’ engagement with performance analysis. International Journal of Performance Analysis in Sport, 18(2), 350-366. https://doi.org/10.1080/24748668.2018.1477026

Laird, P., & Waters, L. (2008). Eyewitness recollection of sport coaches. International Journal of Performance Analysis in Sport, 8(1), 76-84.

Mitsuhashi, D. (2002). A difference of the subjective analysis and objective analysis in the technical factor of tennis : An examination of the usefulness of a score sheet. Bulletin of Tokaigakuen University, 7, 183-193.

O’Donoghue, P. (2005). Normative Profiles of Sports Performance. International Journal of Performance Analysis in Sport, 5(1), 104-119. https://doi.org/10.1080/24748668.2005.11868319

Reid, M., Morgan, S., & Whiteside, D. (2016). Matchplay characteristics of Grand Slam tennis: implications for training and conditioning. Journal of Sports Sciences, 34(19), 1791-1798. https://doi.org/10.1080/02640414.2016.1139161

The R Foundation. The R Project for Statistical Computing. Retrieved August 18 from http://www.r-project.org

Weston, N. J., Greenlees, I. A., & Thelwell, R. C. (2011a). Athlete perceptions of the impacts of performance profiling. International Journal of Sport and Exercise Psychology, 9(2), 173-188.

Weston, N. J., Greenlees, I. A., & Thelwell, R. C. (2011b). The impact of a performance profiling intervention on athletes' intrinsic motivation. Research Quarterly for Exercise and Sport, 82(1), 151-155.