| HOME | HELP | CONTACT US | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Sign In to gain access to subscriptions and/or personal tools.
|
|
|
|||||||
Sign In to gain access to subscriptions and/or personal tools. |
|||||||||
Letter to the Editor |
1 Cincinnati, Ohio2 Newark, Delaware3 Nashville, Tennessee
Dear Editor:
We are writing to comment on "The Drop-Jump Screening Test: Difference in Lower Limb Control by Gender and Effect of Neuro-muscular Training in Female Athletes" by Noyes et al (February 2005, pp 197–207), where methodological errors and bias undermine the validity of the study conclusions. Two experienced sports orthopaedic researchers (K.P.S. and L.S.M.) reviewed the paper using the evidence-based medicine systematic literature review worksheet recently published by Spindler et al in the Journal of the American Academy of Orthopaedic Surgeons. The drop-jump screening test study has subsequently been cited as a reliable and valid test of abnormal lower limb alignment in several papers, most of which were published by the authors of this study in The American Journal of Sports Medicine. While intraclass correlation coefficients (ICCs) demonstrated acceptable reliability, there is no evidence in this paper, or any of the subsequent papers, that the test is valid.
In the study, the authors hypothesized that during a drop-jump: (1) females have decreased knee separation distances upon landing and acceleration; (2) male athletes have neutrally aligned lower limb positions upon landing and acceleration; and (3) a 6-week neuromuscular training program will significantly increase knee separation distance in female athletes.
The main outcome variables were knee separation distance (gender difference), ankle separation distance (gender difference), and how the training regimen changes knee and ankle separation distances in females only. These distances were measured off of a video screen. They concluded that both females and males have valgus alignment and that the females had improved knee separation distance after training.
Methodologically, while there is a comparison group in the study, the design is incomplete; the males were not trained, despite the fact that they had the same identified "abnormal" knee separation distances as the females. In addition, there are no reported data for what appears to have been a control population. Seventeen female subjects who did not undergo training underwent test-retest measurements of the jump-landing sequence taken 7 weeks apart (and ICCs were taken). Neither the test-retest reliability for the knee separation distance nor the data themselves were presented. They should have been. Consequently, the readers do not know if the differences seen in the trained female group are due to test-retest "learning" improvement, natural test-retest variation, or the training regimen.
Several significant sources of potential bias undermine the credibility of this study. First, there was potential selection bias. The population was not stratified by sport or level of play. The female subject population was not randomized for control/treatment group inclusion. The authors state that the female population participated in the treatment group strictly on a voluntary basis. However, the fact remains that referral/recruitment bias most likely exists, because only 62 of 325 athletes participated in the training regimen. The authors state that all female participants tested were given information about the training program and 62 participated in the training. How were they selected? Were they offered the training program in exchange for their participation in the data collection procedures? Did the center pay the athlete? Did the athlete pay to participate in the training regimen?
Second, there was potential exclusion/transfer bias. The authors do not mention why only certain subpopulations were chosen to perform secondary tests (instead of the entire treatment or control populations). Only 54 female athletes were selected to undergo isokinetic tests. Ten subjects were selected for same day test-retest measurements. Seventeen subjects were selected to undergo study protocol 2 times, over a 7-week period.
Finally, there was potential detection bias in the knee separation distance measurement. The research assistant viewed the 3 trials and used the trial that "best represented the athlete’s jumping ability" for measurement. This "best" representation was not operationally defined and provided enormous opportunity for selection bias. Either the average of the 3 trials or the 1st, 2nd, or 3rd jump from all participants should be used. The data analyzer should have been blinded to both gender and index/training time. The authors’ definition of "abnormal" knee separation distance was arbitrary. In addition, there was no validation of this videographic method.
The primary outcome measure, knee separation distance in the drop jump, has not been validated. While a valgus lower limb alignment has been hypothesized to predispose female athletes to anterior cruciate ligament (ACL) ruptures, no studies have demonstrated that, in fact, a valgus alignment measured from 2-dimensional video does predispose individuals to injury. In addition, the authors reference the term "valgus" on 17 occasions, but they never define "valgus" or the relationship of knee separation distance to "valgus." In all of the studies performed by this group where the drop-jump test was used, the vast majority of their subjects in this study and all subsequent studies (eg, male, female, prepubertal, postpubertal) have normalized knee separation distances of less than 60%. These percentile groups were chosen based on no evidence. As the authors state, "These percentile groups were chosen arbitrarily, but we believed that 60% represented a distinctly abnormal lower limb valgus alignment in the coronal plane, which was visually evident from the test photographs." The fact that >70% of all subjects tested and reported in the papers recently published by this group have, by the authors’ own arbitrary definition, "abnormal knee separation distances" renders the test questionable. Why would the authors pick a value that by statistical definition would be normal and call it abnormal? There is no evidence that the 2-dimensional representation captures any at-risk movement, and given that the vast majority of subjects are abnormal by their own arbitrary definition presents a problem for this and the other studies. Since we do not know what the test represents, we do not know what abnormal values are and we do not have evidence that this measurement has any inferential power for predicting risk for lower extremity injury in females or males. What does it screen for and why should the readers be interested that it is or is not different between boys and girls and that it is or is not affected by training?
We are concerned with the persistent citing of this drop-jump test as a valid and reliable test of abnormal lower limb alignment and its inferential use to predict those at risk for ACL injury. The arbitrary categories, inherent biases in this original citation, and methodological flaws require much further study before this test can be shown to be a useful clinical test.
4 Cincinnati, Ohio
We are pleased to have the opportunity to respond to the comments raised by Hewett et al regarding our article "The Drop-Jump Screening Test: Difference in Lower Limb Control by Gender and Effect of Neuromuscular Training in Female Athletes," which was published in The American Journal of Sports Medicine in February 2005. The authors also refer to "subsequent papers" that, although not referenced, we assume to be "Assessment of Lower Limb Neuromuscular Control in Prepubescent Athletes" (
Am J Sports Med. 2005;33,1853–1860
The authors correctly stated the hypotheses from our original prospective study. We were therefore surprised that they found that the fact that male participants were not trained as methodologically incorrect. It was not a purpose or hypothesis of our study to assess a training effect in male participants. In fact, we could not locate via a PubMed search any study by Hewett, Synder-Mackler, or Spindler that assessed an effect of a neuromuscular training program in male athletes. The current focus of the issues of noncontact ACL injuries and ACL prevention programs has been on reducing this problem in the female athlete.4 We believe that our methods in terms of the population selected were sound according to our stated purposes and hypotheses.
Secondly, the authors questioned the subpopulation selected for the reliability studies conducted in our first investigation. It is a well-established practice to take a random sample from a large cohort to conduct reliability analyses.2,5,13 It is also well established that ICCs greater than 0.70 represent adequate reliability.11 Hewett et al stated, "Neither the test-retest reliability for the knee separation distance nor the data themselves were presented." On pages 199–200 of our investigation10 we wrote, "For the test-retest trial, the ICCs for the hip separation distance demonstrated high reliability (preland, 0.96; land, 0.94; takeoff, 0.94). For the within-test trial, the ICCs for the hip, knee, and ankle separation distance were all >0.90, demonstrating excellent reliability of the video-graphic test and software capturing procedures." Hewett et al questioned whether a training effect could have been present in the test-retest trial or due to the training program. We appreciate the opportunity to clarify this confusion, as the athletes who underwent the test-retest trial did not undergo training, but agreed to undergo a second video drop-jump test for the purposes of our investigation.
In our original investigation, athletes were not sorted according to sport due to the finding that the majority (80% of the female participants and 72% of the male participants) had a marked decrease in knee separation distance (valgus overall lower limb alignment) on landing. We stated that the athletes participated in a variety of sports activities. To further respond to this issue, 228 of the female athletes were participating in volleyball, 40 in soccer, 24 in basketball, 15 in gymnastics, and 18 in multiple sports. We conducted an analysis to determine the percentage of female athletes with a marked decrease in knee separation distance according to each sport to answer this question. We found that 100% of gymnasts, 89% of multisport players, 88% of basketball players, 80% of soccer players, and 75% of volleyball players had such a marked decrease in knee separation distance on landing.
The authors were correct in that randomization was not performed in this investigation for training. As we stated, female athletes were provided with information regarding a 6-week Sportsmetrics training program and participation was strictly voluntary, with no attempt made to influence certain athletes over others to complete training. No athlete was paid or provided extra incentive, hence the use of the word "voluntary." No recruitment or referral bias existed. However, we acknowledge that a motivational bias could have affected the results of the second drop-jump test. Whether the improvement in knee separation distance detected was a result of the rigorous training program or motivational bias cannot be answered.
Hewett et al questioned why we tested "only certain subpopulations" after training in terms of the isokinetic muscular assessment. In fact, we tested (as stated on page 198 of the investigation) 54 of the 62 (87%) trained female athletes, a high and adequate percentage to assess training effects on quadriceps and hamstrings peak torques.
The question of a detection or selection bias can be answered by the high reliability of the drop-jump test methodology. The results of our within-test trial, in which all ICCs of the 3 trials were >0.90, clearly demonstrate that in fact it does not matter which trial is selected as the final test result. One could select any of the 3 trials. It is not possible using the methodology developed for the video software to blind the examiner to gender. We provide every athlete with a printout of the video test result, shown in Figure 1 in the original study10 and in Figure 1 of our third study.1 This is accomplished within 1 week of the test and is done for all athletes, regardless of whether they undergo Sportsmetrics training. Therefore, it is not possible to blind the examiner according to whether the session was done before or after training. The education of athletes, parents, and coaches is paramount in our program, and we provide the results of testing in a timely manner. This includes not only the video drop-jump test, but others that assess flexibility, speed, agility, endurance, and isokinetic muscular indices. We believe that all neuromuscular training programs should include similar pre- and posttesting of these indices to demonstrate if the program is effective in altering neuromuscular deficiencies, as well as reducing the rate of noncontact ACL injuries.
Hewett et al are correct in that we clearly stated that our selection of 
60% normalized knee separation distance arbitrarily represented, in our opinion, a distinctly abnormal lower limb valgus alignment in the coronal plane. The examples provided in all 3 of our investigations show the dramatic differences between knees with 60% and those with >60% normalized knee separation distance (see, for instance, Figures 5 and 6 in our original investigation10). The question of the validity of the test was raised because the majority of the population fell into this category. We believe this is not a question of test validity, but simply a finding that was clearly evident from all of the data, including not only the distribution of athletes into the normalized knee separation distance categories, but also the means and 95% confidence intervals of the normalized knee separation distances. In addition, the test depicted differences (improvement) in the mean absolute and normalized knee separation distances in the trained female athletes. In our original investigation, we acknowledged, "It may be that this single-plane video test was not sensitive enough to depict relevant differences in landing patterns between the sexes."
The assessment of lower limb valgus on landing is an important research area because this is believed to be a common cause (or effect) of ACL rupture. Many authors have expressed that an athlete may be at increased risk for ACL injury if the lower extremity is not properly aligned on landing.3,7,8 We believe future investigations, using sophisticated 2- or 3-dimensional equipment, should provide not only means and standard deviations of these data,5 but a distribution of the data so the readers can understand the spectrum of lower limb alignment on landing from a drop-jump.
Hewett et al question the definition of valgus as used in our investigations. We used the phrases "lower limb axial alignment" and "lower limb valgus alignment position" repeatedly in our papers, indicating that valgus referred to the position of the entire lower extremity, and not the position at the knee joint.
The writers also questioned the relevance of a video drop-jump test in capturing any at-risk movement and "its inferential use to predict those at risk for ACL injury." In response, we wish to reiterate the limitations of our drop-jump video test, which were previously described on page 201 of our investigation: "We developed this test to provide a general indicator of an athlete’s lower limb axial alignment in the coronal plane in a straightforward drop-jump and vertical takeoff task. We do not propose that this test can be used as a risk indicator for knee ligament injury. We recognize that the video analysis only depicts hip, knee, and ankle positions in a single plane during one maneuver and that noncontact ACL injuries may occur in side-to-side or cutting motions."10
We disagree with studies in which knee valgus has been designated as a predictor of ACL injury when many other potential risk factors were not also measured or assessed.6 No study has provided conclusive evidence of coronal, sagittal, and axial motion or moment abnormalities (or differences between genders) in the lower limb as a whole or for the hip, knee, and ankle joints separately during jumping tasks. The appropriate longitudinal study of ACL risk factors must include not only neuromuscular indices, but anatomic, environmental, and hormonal factors as recently described by Griffin et al.4 And there must be enough ACL injuries that subsequently occur in the population to avoid a type II error.12 To date, this study has not been conducted and (understandably) considerable disagreement exists in the medical community regarding this issue. In fact, the finding that knee valgus is a significant risk factor for ACL injury was not supported by the recently published Consensus Statements from the 2005 Hunt Valley II Meeting, which only stated, "We need to better understand the role of knee valgus, or perhaps more precisely, ‘apparent knee valgus’ on ACL injury rates."4
Not only did we state in our original investigation that our video drop-jump test was not to be used as a risk indicator for knee ligament injury, but we also discussed the multitude of other variables that may influence ACL ruptures in female athletes (page 20510), including both intrinsic and extrinsic factors.
Our finding that a neuromuscular training program significantly improved lower limb valgus alignment (knee and ankle separation distances), hamstrings peak torques, and hamstrings-quadriceps ratio is important. While other neuromuscular training programs have been reported to reduce the incidence of ACL injury in female athletes, the reasons for these findings were unclear, as athletes were not tested before and after participation.9 We believe that athletes who demonstrate poor overall lower limb alignment on landing and poor hamstring and hip muscle strength are generally at increased risk of a knee ligament injury. However, we and others acknowledge the lack of blinded, controlled, randomized trials in this area.4 The multitude of unanswered questions regarding ACL injury risk factors and knee injury prevention programs was documented by Griffin et al4 and demonstrate that there is clearly a tremendous amount of work left to be done in this area.
REFERENCES
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | CONTACT US | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
