Development and validation of a brief general and sports nutrition knowledge questionnaire and assessment of athletes’ nutrition knowledge

The research was approved (S16/267) by the La Trobe University’s SHE College Human Ethics Sub-Committee (SHE CHESC). All participants were provided with the participant information statement and consent form (online) and ‘agreed’ to participate (electronically).

The data used to develop the abridged version of the NSKQ was the same data that was collected for the final stages of the full-length NSKQ validation. Participants (n = 181) were Australian athletes and university students, recruited from November 2016 to January 2017 [9]. The sample was predominately female (75%), aged 17–25 (52%), born in Australia (81%), and university educated (96%) (Additional file 1: Table S1).

Item analysis (step 6) was performed using Rasch analysis only. Rasch analysis was conducted using RUMM2030 Professional software. The aim of this step was to remove items that were causing misfit to the Rasch model. Misfit was assessed based on overall item-interaction, overall person-interaction and overall item-trait interaction. A SD of 1 and mean of 0 for overall-item and overall-person interaction, and non-significant Chi-Square probability for overall item-trait interaction indicate compliance with the Rasch model (i.e. no misfit). In order to determine which items (or persons) were causing misfit, the following indicators were evaluated: individual item fit residuals, item characteristic curves (ICC), category probability curves (CPC) and differential item functioning (DIF). Detailed definitions and interpretation of the aforementioned statistics are beyond the scope of this report, but are described elsewhere [6, 12] and summarised in Additional file 1: Table S2. Stricter criteria were applied during item analysis for validation of the A-NSKQ. During the validation of the full-length NSKQ the researchers retained certain items that did not meet one or two of the aforementioned Rasch indicators because they were deemed relevant in terms of assessing gaps in NK and retaining content validity. However, in the validation of the A-NSKQ, all items that failed on a single indicator were excluded.

Assessment of internal reliability and dimensionality (step 7) were also performed using Rasch analysis only for the validation of the A-NSKQ. Reliability was assessed using the PerSepIndex, a summary statistic produced by RUMM 2030 that is analogous to Cronbach α. Both PerSepIndex and Cα are based on repeated split-half reliability assessment; a value of ≥0.7 is an acceptable measure of internal reliability [12]. Dimensionality was assessed using the perc5% statistic; values of 5% or less indicate unidimensionality, a requirement of the Rasch model [20].

Gathering of new data to re-examine the scale (step 8) was not conducted. Rather, a ‘proxy score’ for each respondent was calculated by subtracting the deleted NSKQ items from participants’ original score. Similar methods were used by past researchers [22] to compare scores on the original and revised General Nutrition Knowledge Questionnaire (GNKQ/R-GNKQ). Using the proxy scores, construct validity was assessed using the known-group comparisons method and temporal stability was assessed using the test-retest reliability method (Table 1). For the known-group comparisons assessment, NK (proxy) scores of individuals with and without a formal nutrition education (university course, university subject, diploma, and online course) were compared; statistically significant differences are indicative of construct validity [6]. For the test-retest reliability assessment the correlation of NK (proxy) scores on two attempts of the questionnaire, taken ten days to two weeks apart, were assessed; a correlation of ≥0.7 indicates temporal stability [6]. The correlation between NK proxy scores and original scores was also assessed.

After the NSKQ had been revised and the A-NSKQ had been validated, it was administered to Australian non-elite athletes. Members (n = 3951) of one metropolitan AF and netball league in Melbourne, Australia were invited (via email from the league president) to participate in the study by completing the A-NSKQ online. Interested and eligible (residing in Australia, aged 17 years and older) players completed the survey using Qualtrics Research Suite (Qualtrics, Provo, Utah) from May to April 2017.

All analyses (aside from questionnaire validation) were performed in SPSS (Version 23; SPSS, Chicago, IL, USA). Scores on the A-NSKQ and time taken to complete the A-NSKQ were assessed for normality using the Kolmogorov-Simonov statistic. Chi-Square goodness of fit tests were used to evaluate differences in (categorical) demographic characteristics across gender, sport played, highest level of education, previous history of nutrition education, and highest level sport played. Differences in NK scores were assessed using t-test or ANOVA for parametric data, and Mann-Whitney-U-test or Kruskal-Wallis for non-normal data. Significant differences in questionnaire sub-section scores were assessed using paired sample t-test or Wilcoxon-signed-rank-test. The alpha value for most tests was set at P ≤ 0.05. A Bonferroni correction was applied to Kruskal-Wallis tests, and P was set at ≤0.017. Respondents with more than 10% missing data were removed from analysis. Missing responses to items were coded as incorrect.

The original NSKQ consisted of 89 items. Rasch analysis showed that three items had high fit residuals, 19 items had problematic CPCs, 38 items had problematic ICCs, and one item showed DIF (for country). Nine items failed on two indicators, therefore, in total, 52 items were problematic and were excluded from the A-NSKQ, leading to a 37-item questionnaire.

Individuals who had undertaken nutrition studies scored better in the GNK and SNK sub-sections (nutrition students = 77% versus non-nutrition students = 60%, P < 0.001 [GNK] and nutrition students = 60% versus non-nutrition students = 40%, P < 0.001 [SNK]), indicating construct validity. Test re-test reliability was demonstrated based on the proxy scores (r = 0.8, P < 0.001 [GNK] and r = 0.7, P < 0.001 [SNK]). There was a strong, positive correlation (r = 0.9, P < 0.001) between individuals’ score (%) on the NSKQ and their proxy score on the A-NSKQ. This indicates the removal of the items did not alter overall total (%) NK scores.

Two-hundred-and-seventy-three athletes followed the link to complete the A-NSKQ (response rate = 7%). The individual response rate of the NSKQ could not be calculated for non-elite athletes, because the questionnaire was distributed by a second party and the total number of invitations was not known; the response rate for team presidents who agreed to distribute the NSKQ was 8–13% (Additional file: Table S2). 1

After deleting responses of individuals who followed the link but did not agree to the participant information and consent form (n = 65; not included in completion rate calculations), incomplete responses (n = 22) and respondents who did not meet the eligibility criteria for age (n = 9), there were 177 usable responses. The completion rate for the A-NSKQ (85%) was higher than the average completion rate for the NSKQ (54%); completion rate calculations were undertaken in the same manner (Additional file 1: Table S2).

The median time taken to complete the A-NSKQ was 12 min, compared to 25 min taken to complete the NSKQ.

There were no differences in age, gender, country of birth, level of education and previous nutrition study between those who reported playing at the elite and non-elite level. However, individuals who reported playing a sport other than AF as their primary sport were more likely to play at higher levels of competition (X² = 104.526, df = 1, P < 0.001). Females were more likely to play a sport other than AF (X² = 4.637, df = 1, P = 0.031) and to be university educated (X² = 0.337, df = 1. P < 0.001).

The 37-item A-NSKQ can be completed in around half the time taken to complete the NSKQ (12 versus 25 min). The A-NSKQ covers the same key topics assessed in the NSKQ (weight management, macronutrients, micronutrients, supplementation, sport nutrition, and alcohol);however, it does not evaluate some aspects of these topics, such as the role of specific supplements, and the consequences of dehydration. Therefore, the content and face validity of the A-NSKQ should be reviewed by individuals planning on using the tool, to ensure that it serves their intended purpose.

The results of this study show that the A-NSKQ sub-sections (GNK and SNK) fit the Rasch model, indicating that the items meet the requirements that difficult questions are less likely to be answered correctly and individuals with higher levels of NK are more likely to perform well [20]. Likewise, the questionnaire demonstrated test-retest reliability and achieved or approached pre-determined cut-off values for internal reliability. However, a limitation is that these assessments were based on proxy scores. Construct validity was also determined based on the proxy scores, and was confirmed when the A-NSKQ was administered to non-elite athletes, as those with previous nutrition education scored significantly higher than those without any formal nutrition education (Additional file 1: Table S5). The criterion validity of the A-NKSQ could be assessed in future studies by administrating both questionnaires to participants and calculating the correlation between scores (%). However, it may be difficult to achieve adequate response rates for this step, as respondents are likely to find completing both tools burdensome. In addition, participants may research the correct answers to questions before the second completion, which would influence the validity of this method.

The invitation to complete the A-NSKQ was distributed to a large number of non-elite athletes via email, but response rates were low (7%). Although web-surveys are known to have lower response rates than mail-based surveys, these usually sit around 34% [23]. Response rates for NSKQ could not be calculated because it was distributed via Facebook groups and online forums, or via second parties where total exposure was not known. However, the percentage of non-elite teams who responded positively to be involved in data collection using the NSKQ (6–13%) was very similar to the response rates of individuals who participated in data collection using the A-NSKQ (7%). This is in line with some reports in the literature that questionnaire length has only a small effect on response rates [18]. Although the shorter length did not appear to have a positive effect on response rates, it did have a positive effect on completion rates. Completion rates in the present study were 85%, which is higher than completion rates (54%) achieved during the validation and use of NSKQ to collect data. As above, this finding is akin to what has previously been reported regarding questionnaire length and completion rates [17]. Future studies should consider distribution methods and survey design to ensure that response and completion rates are optimised. These include designing personalised invitations and sending reminders and notifications if responses have not been obtained [16]. However, personal interest in the topic is one of the key factors influencing individuals’ decision to participate in a survey, and this cannot be controlled by investigators [16].

The results indicate there is room for improvement in non-elite AF and netball players’ NK. Athletes’ mean score was 47%; similarly, non-elite, male AF players scored 51% on the NSKQ ¹. In the present study, athletes performed much better on questions of GNK than SNK, which is akin to results reported by Devlin et al. [24] and Kunkel et al. [25], but in contrast to results reported by Harrison et al. [26] and Barr [27]. Respondents answered items on hydration (q27) supplements (q35, q26, and q37) and micronutrients (q13, q 23, and q25) poorly. These items were also answered poorly when the NSKQ was administered to non-elite AF players¹. In the existing literature there are inconsistencies regarding gaps in knowledge with some studies reporting hydration [28], supplementation [29] and micronutrients [28] were better understood than other topics, and others finding the opposite to be true [26, 30].

A literature review undertaken by Trakman et al. [31] reported that 64% of studies on athletes’ and coaches’ NK found that prior nutrition knowledge, higher levels of education, previously undertaking a nutrition course or currently majoring in nutrition studies correlated with higher NK scores. Ten of fifteen studies that assessed the effects of gender on NK stated there were no significant differences between males and females. All studies that evaluated sporting calibre and type of sport reported these had no effect on NK scores. The present study reflected these findings; previous nutrition education and higher levels of education positively affected NK score, but a relationship between score and sport played, sporting level and gender were not detected. Older participants had better GNK, which is in contrast to findings in other cohorts [32]. Of note, however, the current sample had only a small proportion of athletes from sports other than AF, and playing at the sub-elite or elite level.

Additional file 1:Table S1. Characteristics of participants who completed the NSKQ for validation (n = 181). Table S2. Summary of RUMM2030 statistics that are assessed to determine reason for misfit to the Rash model. Table S3. Response and completion rates for the NSKQ and A-NSKQ. Table S4. Responses (percent correct) of individual items in the A-NSKQ. Table S5. Differences in NK on the A-NSKQ based on participant characteristics. (DOCX 34 kb)