Runners are at relatively high risk for sustaining an overuse injury. While many risk factors have been documented so far, previous reviews have mostly failed to identify effective interventions to lower injury risk in runners. To review the high-quality evidence on two types of preventive interventions – movement therapy interventions and training-modification interventions, regarding running-related injury prevention. Pubmed (MEDLINE), PEDro and Cochrane Central Register of Controlled Trials databases were searched in April 2017, with no date or language restrictions, using following search terms: running injury prevention, running injury therapy, running injury incidence, running injury exercise and running injury risk. Studies were included if they were a randomized controlled trial or prospective cohort study, investigated the effects of movement therapy or training modification interventions, contained a population of runners or other populations, involved in running (e.g. military recruits), and reported lower extremity injury incidence rates specific to running. In total, 4935 citations were identified, 69 of which were retrieved for full-text evaluation. Seven articles met the inclusion criteria and were included in meta-analysis. Two separate meta-analyses were carried out for both intervention types. First meta-analysis showed no preventive effects of movement therapy interventions, with an overall risk ratio of 0.98 (p = 0.81, I2 = 42 %). The second meta-analysis showed no overall preventive effect of training modifications, with an overall risk ratio of 0.78 (p = 0.35, I2 = 79%). No evidence was found to support the preventive effects of movement therapy or training modification. This may primarily be due to non-optimal intervention designs, such as using inappropriate placebo exercises. Preventive programs may also be more effective when carried out prior to running program onset.
There is a shortage of high-quality evidence to support that movement therapy and training modification can reduce running-related injury risk.
Ineffectiveness of some interventions may be attributed to poor study design (e.g. inappropriately chosen exercises).
Interventions in future research should be based on previously identified risk factors and rare previous successful interventions.
Running, as other physical activities, has many positive effects on health and well-being. Endurance running was shown to induce positive changes regarding body composition, resting heart rate, aerobic capacity and lipid profile (Hespanhol Junior et al., 2015). On the other hand, runners are at relatively high risk for sustaining an injury, particularly for various overuse injuries. A survey of 4 385 male runners carried out back in 1984 showed that 45.8% of them have sustain an injury in previous year, with 14.2% requiring medical care (Marti et al., 1988). A more recent systematic review reported the incidence of lower extremity running injuries ranging from 19.4% to 79.3%, with knee being affected most often (van Gent et al., 2007). Videbæk et al. (2015) reported the incidences of 17.8 and 7.7 per 1000 hours of exposure in novice runners and recreational runners, respectively. Common running injuries include medial tibial stress syndrome, Achilles tendinopathy, patellar tendinopathy, plantar fasciitis, ankle sprain, iliotibial band syndrome and patellofemoral pain syndrome (Lopes et al., 2012). As running grows more and more popular every year, the knowledge of prevention strategies is as desired as ever.
Many risk factors for sustaining a running-related injury have been documented. In their review, Saragiotto et al. (2014) found previous injury to be the main predictor for future injuries. Others were weekly distance, weekly training frequency and increased Q-angle (the angle between femur and tibia). Even more risk factors are known for specific injuries. Newman et al. (2013) reported prior use of orthotics, fewer years of running experience female gender, previous history of medial tibial stress syndrome, increased body mass index and navicular drop to increase the risk for sustaining a medial tibial stress syndrome. Neal et al. (2016) found the association between patellofemoral pain and peak hip adduction and internal rotation, contralateral pelvic drop and reduced peak hip flexion. Goff and Crawford (2011) listed excessive foot pronation (pes planus), excessive running volume, high arch (pes cavus), leg length discrepancy, obesity, prolonged standing/walking occupations (e.g., military personnel), sedentary lifestyle and tightness of Achilles tendon and intrinsic foot muscles to contribute to plantar fasciitis development. Regarding Achilles tendinopathy, Rutland et al. (2010) reported several intrinsic (strength imbalances, postural malalignment, lack of strength and flexibility, limited dorsiflexion range of motion) and extrinsic (non-gradual training program, training surface, etc.) risk factors in their review.
One of the most comprehensive literature reviews on preventing running injuries to date is by Yeung et al. (2011). Focusing only on lower limb soft-tissue injuries, the only efficacious strategies were wearing a patellofemoral brace for preventing anterior knee pain (two trials) and utilizing custom-made foot orthoses for reducing MTSS in military recruits. Craig et.al (2008), similarly found the use of “shock-absorbing” insoles as the only effective strategy for preventing medial tibial stress syndrome. Enke and Gallas (2012) focused on prevention and management of common running-related injuries and concluded that the knowledge in this area is very limited and recommended using individualized treatment instead of a generalized prevention program for now. Even footwear choice seems to have little effect on running-related injury risk (Knapik et al., 2014).
Running-related injuries are often serious enough to cause a cessation of training and were shown as the most frequent reason (31%) for abandoning running in study of cohort of runners. The main objective of our meta-analysis was to assess the current knowledge on two types of preventive interventions – movement therapy interventions and training-modification interventions, and provide recommendations for clinicians regarding preventive program design for runners and point out how researchers should approach the problem in the future.
An electronic search of the PubMed (MEDLINE), PEDro and Cochrane Central Register of Controlled Trials databases was performed in April 2017 for randomized controlled trials and prospective cohort studies, examining the associations between movement therapy interventions or training modifications and running-related injury risk. We imposed no date or language restriction. We used the following search terms: running injury prevention, running injury therapy, running injury incidence, running injury exercise and running injury risk. In the MEDLINE database, we used set operators to search with the following combination: running injury AND (prevention OR therapy OR exercise OR incidence OR risk OR rehabilitation). After the initial search, titles and abstracts were screened to identify potentially relevant articles. Afterwards, full texts were obtained and final selection was made upon reading those. Both authors carried out all steps of article collection independently. Potential disagreements between authors were resolved by discussion and additional revision.
Studies were included in the meta-analysis if they met the following criteria: a) the study design was a randomized controlled trial or prospective cohort study b) the study investigated the effects of either movement therapy intervention (stretching, any type of resistance training, balance/stability training, coordination training or combination of those) or training-modification intervention (manipulation of training volume, intensity and frequency or adding substitute training) on running-related injury incidence, c) study subjects had to be involved in running – professionally, recreationally or as a part of their job (military personnel), d) running-related injury incidence was reported.
Both authors carried out the data extraction independently. The total number of running-related injuries of lower-limb in experimental and control groups were the main outcomes and were extracted to be pooled into a meta-analysis (two separate analyses were carried out for either type of intervention). All articles contained a descriptive display of injury types; therefore, extraction of only running-related injuries was possible. Other data extracted included: authors, year of publication, participants` characteristics (mean age, gender and experience) and intervention characteristics (duration, type and volume).
Assessment of methodological quality of included studies
Both authors independently assessed risk of bias of included studies using the PEDro Scale, which was shown to provide a fairly reliable scores of methodological quality of randomized controlled trials (Maher et al., 2003). A point was awarded for each of 11 criteria if it was clearly satisfied. Potential disagreements between authors were resolved by discussion and additional revision.
I-squared (I2) test and chi-squared (Chi2) test scores were calculated to assess the statistical heterogeneity among studies. For I2 interpretation, we used the following criteria: 0% to 40% is considered low heterogeneity; 30% to 60% may represent moderate heterogeneity; 50% to 90% may represent substantial heterogeneity; 75% to 100% indicates considerable heterogeneity. Results for Chi2 were considered statistically significant at p < 0.1. We followed the guidelines for identifying and measuring heterogeneity provided in Cochrane Handbook for Systematic Reviews of Interventions (Higgins and Green, 2011).
Both meta-analyses were conducted using a random effect model (Mantel-Haenszel method). The difference in probability of injury occurrence between groups was expressed as relative risk (risk ratio), with entitled 95% confidence intervals. Heterogeneity assessment and effect size calculations were both performed with RevMan 5.3 (Review Manager, Version 5.3., Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014).
Our search initially yielded 4935 citations. After removing the duplicates and screening the titles and the abstracts, 69 articles were left for full text evaluation. Upon reading full texts, 62 articles were excluded for not meeting inclusion criteria, leaving seven articles for further analysis (Figure 1). Five articles fell into movement therapy category and two into training modification category. There were no disagreements between authors on data extraction.
Effect of movement therapy interventions on running-related injury risk
First meta-analysis showed no preventive effects of movement therapy interventions, with an overall risk ratio of 0.98 (p = 0.81). Only one intervention reduced the injury risk significantly. Overall, there were 341 injuries in the experimental group (n = 2009) and 346 injuries in the control group (n = 1977). Moderate statistical heterogeneity was present between studies according to I2 test (42%), while Chi2 test was not significant (6.88; p = 0.08). The results of meta-analysis are outlined in Figure 2.
Effect of training modification interventions on running-related injury risk
The second meta-analysis showed no overall preventive effect of training modifications. Pooled risk ratio of 0.78 was not statistically significant (p = 0.35). Overall, there were 95 injuries in the experimental groups (n = 420), and 123 injuries in the control groups (n=416). High statistical heterogeneity was present between studies according to I2 test (79%) and Chi2 test (4.97; p = 0.03). The results of the second meta-analysis are outlined in Figure 3.
Methodological quality of included studies
The methodological quality assessment showed a range of 6-8 of 11 points awarded. Authors were able to solve all the disagreements with additional revision of the text and discussion. Summary of the assessment is shown in Table 1.
Interventions and findings of individual studies
Bredeweg et al. (2012) investigated the effects of walking and hopping exercises on incurring a running-related injury. The authors’ theorized that progressive increase biomechanical loads before the onset of a running program may reduce risk of injury. Participants in the experimental group took part in a 4-week preconditioning program. Two session per week consisted of walking and intermediate hopping bouts. Six repetitions of 5-minute walking bouts followed by a set of hopping were performed. Number of hops per set between the first and the last session increased from 50 to 90, respectively. One session per week included walking only. Duration of this session was also progressed (from 30 to 60 minutes). After the preconditioning period, a 9-month running program commenced. The control group did nothing up to this point. Participants ran three times per week at low intensity and were instructed not to perform any stretching exercises at any time. The incidence of injuries were 15.2% in the experimental and 16.8% in the control group. Preconditioning program had no significant effects on running injury risk during a 9-month follow-up period (p = 0.69).
Brushoj et al. (2008) compared the effectiveness of a progressing training program consisting of strength, coordination and flexibility exercises, as compared to a placebo training program, on running-related injuries in military recruits. Both experimental and control group undertook a 3-month military training and concomitantly performed a preventive training program (three sessions per week). Participants in the control group performed placebo exercises, such as biceps curl and pectoralis major stretch. The exercises in the experimental group were the following: squats, lunges, hip rotation/abduction with elastics, forefoot lift, quadriceps stretch and a foot coordination exercise. All exercises were performed in three sets, with load increasing every two weeks. The authors did not report the load, expressed as percentage of one-repetition maximum. The number of repetitions ranged from five to twenty. Incidences in the experimental and the control group were 21.3 % and 17.7 %, respectively. The effect of intervention was not significant (p = 0.162). In both groups, more injuries occurred in earlier stages of the course.
Van Mechelen et al. (1993) investigated the effects of warm-up and cool-down protocol on running injury incidence. During a 16-week running training program, participants in the experimental group performed 6 minutes of low-intensity running, 3 minutes of dynamic stretches and 6 minutes of static stretching prior to every workout. The same activities in reversed order were performed after the workout. Authors did not specify the dynamic stretching exercises used. The static stretching exercises included three bouts (10 seconds each) for the iliopsoas and quadriceps muscles, the hamstrings, and the soleus and gastrocnemius muscles. These static stretching exercises were also performed twice on the rest days. Incidences in the experimental and the control group were 16.4% and 13.8%, respectively. Since the amount of running was tracked, the authors of the original paper were also able to calculate the incidence expressed as number of injuries per 1000 hours of exposure, which were 5.5 in the experimental and 4.9 in the control group. Thus, intervention showed no significant preventive effects.
Pope et al. (2000) investigated the effects of stretching on lower-limb injury incidence among military recruits. During a 12-week training course, both experimental and control group performed warm-up activities before training sessions. Additionally, the stretch group performed six 20-s static stretches for each major lower extremity muscle group - gastrocnemius, soleus, hamstrings, quadriceps, hip adductor, and hip flexor muscle groups. Authors reported an insignificant hazard ratio of 0.95 in favor of the experimental (i.e. stretch) group. For the purposes of our analysis, we calculated the risk ratio, which was 0.81 (95% CI 0.68-0.98; p = 0.02). Thus, the trial showed moderate beneficial effect of stretching.
In a similar study, Pope et al. (1998) investigated whether stretching calf muscle prior to exercise could reduce the risk of running-related injuries in military recruits. Participants in the experimental group (n = 549) stretched the soleus and gastrocnemius muscles under supervision. Two 20-second stretches on for muscles on each side (8 stretches total). The control group (n=544) performed arm stretches of the same volume. Over the 12-week course, there were 23 and 25 injuries recorded in experimental and control group, respectively. The intervention was therefore not effective for lowering injury risk (p = 0.75).
Rudzki (1997) assessed the effectiveness of exposure modification among military recruits on running-related injury. Participants in the experimental group replaced part of their running sessions with weighted marching. The standard program (which control group utilized) included 26.5 km of running over the 12-week course, while the experimental group only did 10 km. The load for weighted marching was 16.2 kg at the beginning and from week five, 2.6 kg was added each weak. The speed of marching was increased from 5 km/h to 7.5 km/h throughout the study. The incidences in the experimental and the control group were 25.4 % and 41.7%, respectively. The total number of lower limb injuries was significantly reduced in the experimental group (risk ratio: 0.61; p = 0.02).
Buist et al. (2008) investigated the effect of a gradual training program on running-related injury in novice, adult runners. Experimental group underwent a thirteen-week gradual training course utilizing the 10% rule for increasing running volume, while controls utilized a standard eight-week training protocol. Running volume increased at a slower rate in the experimental group. Both groups trained three times per week, with weekly training time increasing from 30 to 90 minutes for experimental and from 30 to 95 minutes for control group. Incidence of injuries was similar in both groups – 20.8% in the experimental group and 20.3% in the control group.
An overview of participants’ and intervention characteristics is shown in Table 2.
Neither of the meta-analyses showed significant overall effects. There seems to be a shortage of high-quality evidence to support effects of movement therapy and training modification based interventions in the context of running-related injury prevention. There was substantial heterogeneity between studies, in terms of participant’s characteristics, intervention types and sample sizes. Moreover, we observed moderate to high risk of bias in most of the included studies. Three of the total six trials were done on military recruits. Only in one study (Bredeweg et al., 2012), the intervention was carried out and concluded prior to running program onset (in which the injuries were tracked). In other studies, intervention and running program began and were concluded simultaneously.
Two of the studies (Pope et al., 2000; Rudzki, 1997) in our analysis showed statistically significant preventive effects. Participants were military recruits in both studies. While the stretching intervention by Pope et al. (2000) could be easily utilized by runners, substituting running with weighted marching is questionable. Moreover, caution should be used when extrapolating results from studies conducted on military recruits to civilian runners, since military training includes various other activities, such as hiking, swimming, etc. In addition, the volume of running done by military recruits is relatively low (a single running session does not exceed 8 km), comparable perhaps to low-level recreation runners.
Ineffectiveness of certain interventions in our meta-analysis could perhaps be attributed to study design (exercise choice, intertwining of the intervention and the running program, etc.). For instance, Brushoj et al. (2008) had the control group undertake a placebo exercise program. However, some of the exercises performed could have an effect on running cycle biomechanics, and therefore cannot be considered true placebo. Specifically, exercises for strengthening of the trunk flexors and extensors were implemented in the control group, which may have positive effects on trunk stability and consequentially on running technique. Future interventions should be designed more judiciously.
The low amount of articles found may be attributed, in part, to the demanding and time-consuming design needed when conducting incidence studies. Assembling a large enough sample size, participant injury tracking, and analyzing the effects of potential covariants are some of the potential barriers to study design and implementation. In the review by Yeung et al. (2011), which included broader spectrum of interventions, only three of the total twenty-five trials included strictly civilian recreational runners as research participants (other populations included military recruits, prisoners and soccer referees).
Another consideration regarding trial design is the concomitancy of prevention programs and running. Only in one study (Bredeweg et al., 2012), participants concluded the prevention program and then went on to the running program. In other five studies, intervention and running program were performed simultaneously. The overall higher volume of training may have canceled out the positive effect of the interventions. On the other hand, our goal should be to design an intervention that is applicable either before or during training program. Many runners, especially professionals, may refuse to participate in a running injury prevention program if it ment completely abandoning running for the time being.
A good example of a comprehensive intervention was designed and tested by Sharma et al. (2014) on British Army recruits. This study was excluded from our analysis, because only medial tibial stress syndrome incidence was tracked. The authors identified at-risk recruits via a baseline plantar pressure assessment. The experimental group received running gait retraining, neuromuscular control exercises and flexibility training sessions whereas controls received no interventions. Both cohorts performed an identical military training program. Participants in the experimental group had a reduced relative risk of developing MTSS versus controls. Future, prospective investigations conducted in a civilian population are warranted to assess the generalizability of these findings.
We believe that the best clinical approach for now is to identify and directly treat the major risk factors for running-related injuries of each individual. Much more evidence is present for effectiveness of interventions to target specific risk factors. For instance, intervention by Snyder et al. (2009), which consisted of three single-legged exercises (two hip rotations and pelvic rotation) significantly lowered the level of foot pronation during stance phase. Ground reaction forces can be substantially lowered with gait retraining methods (Crowell and Davis, 2011). To form a universal preventive exercise program for runners, more randomized controlled trials or prospective cohort studies are being desired to evaluate the effect of interventions on running-related injury risk. To begin with, we again recommend revisiting trials evaluating risk factors for running-related injuries. Findings of such studies should serve as basis for designing interventions for further trials. Secondly, studies investigating the effects of various interventions on previously identified risk factors should be reviewed and more should be designed and carried out. Integration of findings from both types of studies mentioned should lead to a design of comprehensive, judiciously designed interventions, which should then be tested with incidence studies.
A strong limitation of our meta-analysis should be acknowledged. There was substantial heterogeneity between studies. As noted before, half of the included studies investigated military recruits. In one study, all participants were male. Intervention time also differed between trials. Therefore, our meta-analysis provides an overview of the current evidence at best. Estimated overall effects of both analyses should be interpreted cautiously, despite using random effects model. Any conclusions should not be generalized to runners in general. This review only focused on high quality studies, while there may be other effective interventions tested with a different design. Some studies that were excluded form our analysis showed promising results (Sharma et al., 2014; Snyder et al., 2009). For now, clinicians should look for effective preventive methods in other type of trials and different interventions.
The current meta-analysis does not provide high-quality evidence to support the effectiveness of movement therapy or training modification based interventions for preventing running injuries. Further randomized controlled trials and prospective cohort studies are desired and should be based on the previously acquired knowledge on eliminating risk factors for sustaining running-related injuries. Individual treatment should be used in clinical practice until an effective generalized preventive program is established.
The authors declare that they have no conflict of interest.
Post-graduate student, University of Primorska, Faculty of Health Sciences, Polje 42, 6310 Izola,
interests: Musculoskeletal injury prevention, physical conditioning, ergonomics.
Bredeweg S.W., Zijlstra S., Bessem B., Buist I. (2012) The effectiveness of a preconditioning programme on preventing running-related injuries in novice runners: a randomised controlled trial. British Journal of Sports Medicine 46, 865-870.
Brushoj C., Larsen K., Albrecht-Beste E., Nielsen M.B., Loye F., Holmich P. (2008) Prevention of overuse injuries by a concurrent exercise program in subjects exposed to an increase in training load: a randomized controlled trial of 1020 army recruits. The American Journal of Sports Medicine 36, 663-670.
Buist I., Bredeweg S.W., van Mechelen W., Lemmink K.A., Pepping G.J., Diercks R.L. (2008) No effect of a graded training program on the number of running-related injuries in novice runners: a randomized controlled trial. The American Journal of Sports Medicine 36, 33-39.
Craig D.I. (2008) Medial tibial stress syndrome: evidence-based prevention. Journal of Athletic Training 43, 316-318.
Crowell H.P., Davis I.S. (2011) Gait retraining to reduce lower extremity loading in runners. Clincal Biomechanics 26, 78-83.
Enke R. C., Gallas J. E. (2012) Diagnosis, Treatment, and Prevention of Common Running Injuries. Journal of Clinical Outcomes Management 19, 86-94.
Goff J.D., Crawford R. (2011) Diagnosis and treatment of plantar fasciitis. American Family Physician 84, 676-682.
Higgins J.P.T., Green S. (2011) Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. The Cochrane Collaboration. www.handbook.cochrane.org.
Hespanhol L.C., Pillay J.D., van Mechelen W., Verhagen E. (2015) Meta-Analyses of the Effects of Habitual Running on Indices of Health in Physically Inactive Adults. Sports Medicine 45, 1455-1468.
Knapik J. J., Trone D. W., Tchandja J., Jones B. H. (2014) Injury-reduction effectiveness of prescribing running shoes on the basis of foot arch height: summary of military investigations. Journal of Orthopaedic & Sports Physical Therapy 44, 805-812.
Lopes A.D., Hespanhol L.C., Yeung S.S., Costa L.O. (2012) What are the main running-related musculoskeletal injuries? A Systematic Review. Sports Medicine 42, 891-905.
Marti B., Vader J.P., Minder C.E., Abelin T. (1988) On the epidemiology of running injuries. The 1984 Bern Grand-Prix study. The American Journal of Sports Medicine 16, 285-294.
Neal B. S., Barton C. J., Gallie R., O’Halloran P., Morrissey D. (2016) Runners with patellofemoral pain have altered biomechanics which targeted interventions can modify: A systematic review and meta-analysis. Gait & Posture 45, 69-82.
Newman P., Witchalls J., Waddington G., Adams R. (2013) Risk factors associated with medial tibial stress syndrome in runners: a systematic review and meta-analysis. Open Access Journal of Sports Medicine 4, 229-241.
Noehren B., Hamill J., Davis I. (2013) Prospective evidence for a hip etiology in patellofemoral pain. Medicine & Science in Sports and Exercise 45, 1120-1124.
Pope R., Herbert R., Kirwan J. (1998) Effects of ankle dorsiflexion range and pre-exercise calf muscle stretching on injury risk in Army recruits. The Australian Journal of Physiotherapy 44, 165-172.
Pope R.P., Herbert R.D., Kirwan J.D., Graham B.J. (2000) A randomized trial of preexercise stretching for prevention of lower-limb injury. Medicine & Science in Sports and Exercise 32, 271-277.
Rudzki S.J. (1997) Injuries in Australian Army recruits. Part I: Decreased incidence and severity of injury seen with reduced running distance. Military Medicine 162, 472-476.
Rutland M., O’Connell D., Brismee J.M., Sizer P., Apte G., O’Connell J. (2010) Evidence-supported rehabilitation of patellar tendinopathy. North American Journal of Sports Physical Therapy 5, 166-178.
Saragiotto B.T., Yamato T.P., Hespanhol L.C., Rainbow M.J., Davis I.S., Lopes A.D. (2014) What are the main risk factors for running-related injuries?. Sports Medicine 44, 1153-1163.
Sharma J., Weston M., Batterham A.M., Spears I.R. (2014) Gait retraining and incidence of medial tibial stress syndrome in army recruits. Medicine & Science in Sports and Exercise 46, 1684-1692.
Snyder K. R., Earl J. E., O’Connor K. M., Ebersole K. T. (2009) Resistance training is accompanied by increases in hip strength and changes in lower extremity biomechanics during running. Clinical Biomechanics 24, 26-34.
Van Gent R. N., Siem D., van Middelkoop M., van Os A. G., Bierma-Zeinstra S.M., Koes B.W. (2007) Incidence and determinants of lower extremity running injuries in long distance runners: a systematic review. British Journal of Sports Medicine 41, 469-80.
Van Mechelen W., Hlobil H., Kemper H.C., Voorn W.J., de Jongh H.R. (1993) Prevention of running injuries by warm-up, cool-down, and stretching exercises. American Journal of Sports Medicine 21, 711-719.
Videbæk S., Bueno A.M., Nielsen R.O., Rasmussen S. (2015) Incidence of Running-Related Injuries Per 1000 h of running in Different Types of Runners: A Systematic Review and Meta-Analysis. Sports Medicine 45, 1017-1026.
Žiga Kozinc, Nejc Šarabon,
Effectiveness of Movement Therapy Interventions and Training Modifications for Preventing Running Injuries: A Meta-Analysis of Randomized Controlled Trials.
Journal of Sports Science and Medicine(16), 421 - 428.
| Copyright 2001-2017 | All rights reserved. | LEGAL
NOTICES | Publisher
It is forbidden the total or partial reproduction
of this web site and the published materials, the treatment of its database,
of transition and for any means, either electronic, mechanic or other
methods, without the previous written permission of the JSSM.