The validity and reliability of an instrument to assess children's outdoor play in various locations

1. Introduction 

The activities performed by children in their free time after school and on the weekends may make a major contribution to children's physical activity. While several measures have been developed to assess children's overall physical activity,1 there is a lack of reliable and valid measures to assess children's outdoor playtime and the locations in which they play. Previous studies have developed and tested the validity of a parental proxy-report measure of the time pre-school-aged children spend playing outdoors; however, that measure has not been validated in primary school-aged children, amongst whom opportunities to engage in outdoor play are likely to be quite different.2 Other studies have assessed use of recreational time in school-aged children; however, the actual outdoor locations in which children play have not been measured.1 The aim of this study was to develop and test the reliability and validity of survey items that examine the frequency with which primary school-aged children play in particular outdoor locations.

2. Methods 

Information obtained from earlier studies3 in conjunction with a review of the relevant literature was used to formulate a number of proxy-report survey items. These items required parents to report the number of days their child spent playing in eight specified locations (yard at home, friend's/neighbour's yard, own street/court/footpath, nearby streets/courts/footpaths, park/playground, facilities or sports ovals, school grounds, and other places) out-of-school hours on weekdays and weekend days during a typical week. Weekday responses were based on a five-point scale ranging from never/rarely to 5 days per week, and weekend day responses were on a six-point scale ranging from never/rarely to every Saturday and Sunday. Parents were asked to count only the days where their child spent at least 10min in a specific location. Ten minutes was considered an appropriate minimum cutpoint, as according to adult physical activity recommendations bouts of 10min or more are considered beneficial to health.4

The log book presented parents with a 7-day diary where they were asked to indicate on each of the nominated 7 days whether their child had played in specified outdoor locations after-school hours for at least 10min. The locations were the same as those included in the survey.

In order to determine whether the survey items were a reliable measure of frequency of outdoor play, a test–retest reliability study was conducted with a sample of parents of children in grades 3–4 attending one of three primary schools in metropolitan Melbourne. Fifty-three parents provided informed consent and completed the survey on two occasions, 2 weeks apart. The average age of the child about whom the parent responded was 9.5 years (range: 8.3–12.3 years; S.D.=0.73), with 31 boys and 22 girls.

The validity study involved 46 parents with children in grades 3 and 4 who attended a primary school in metropolitan Melbourne, completing firstly the log book over a 1-week period, and then the survey 2 weeks later. The average age of the child about whom the parent responded was 9.2 years (range: 7.9–11.7 years) and there was an equal split of boys and girls. Both studies were approved by Deakin University Ethics Committee and the Department of Education Victoria.

The reliability of the survey items were examined using one-way single measure intra-class correlation coefficient for continuous variables (ICC). ICC values >0.75 were considered indicative of good reliability, 0.50–0.75 moderate and <0.50 poor reliability. The kappa statistic (κ) and percent agreement between responses were used to assess validity. Kappa values were defined as fair to poor (0.00–0.40), moderate (0.41–0.60), substantial (0.61–0.80), and excellent (0.81–1.0). Percent agreement values greater than 66% were classified as fair.5

3. Results 

Reliability results (Table 1) showed that five out of eight weekday items had at least moderate reliability (ICC=0.51–0.82). For the weekend day items, two out of eight items showed good reliability (ICC>0.75), and four items showed moderate reliability (ICC=0.58–0.64).

Table 1. Test–retest reliability study of the proxy-reported frequency with which children play in various outdoor locations.

		% children playing at least once on weekdays or weekend days	ICCa	P
	Weekday Items
	Yard at home	88.7	0.80	<0.001
	Friend's/neighbour's yard	32.7	0.70	<0.001
	Own street/court/footpath	25.5	0.82	<0.001
	Nearby streets/court/footpath	0	0.40	0.002
	Park/playground	32.7	0.63	<0.001
	Facilities or sports ovals where your child can be active	24.0	0.48	<0.001
	School grounds for free play outside school hours	2.0	0.51	<0.001
	Other places where your child can be active (e.g., bush)	21.2	0.47	<0.001
	Weekend day items
	Yard at home	86.5	0.58	<0.001
	Friend's/neighbour's yard	27.0	0.77	<0.001
	Own street/court/footpath	21.6	0.76	<0.001
	Nearby streets/court/footpath	2.0	0.33	0.010
	Park/playground	13.5	0.64	<0.001
	Facilities or sports ovals where your child can be active	23.5	0.63	0.001
	School grounds for free play outside school hours	2.0	0.18	0.105
	Other places where your child can be active (e.g., bush)	11.5	0.62	<0.001

a Intra-class correlation.

Validity results showed that across all weekday and weekend day items (Table 2), seven items had moderate validity according to κ (0.41–0.60). For two weekday items (‘school grounds’ and ‘other places’) the kappa statistic could not be computed as one of the cells contained zero responses. For the weekend day ‘school grounds’ item, κ could not be calculated due to 100% percent agreement between the two measures. Percent agreement between the log book and survey was acceptable for all items.

Table 2. Validity of the proxy-reported frequency with which children play in various outdoor locations.

		Children playing at least once on weekdays or weekend days (%)	Kappa (n=46)	P	Agreement (%)
	Weekday Items
	Yard at home	95.7	0.48	<0.001	63.0
	Friend's/neighbour's yard	41.3	0.40	<0.001	65.2
	Own street/court/footpath	56.5	0.51	<0.001	67.4
	Nearby streets/court/footpath	30.4	0.60	<0.001	80.4
	Park/playground	25.1	0.39	0.007	73.9
	Facilities or sports ovals where your child can be active	32.6	0.35	0.008	67.4
	School grounds for free play outside school hours	19.6	a	–	67.4
	Other places where your child can be active (e.g., bush)	2.2	a	–	86.9
	Weekend day items
	Yard at home	87.0	0.44	0.002	71.7
	Friend's/neighbour's yard	21.7	0.50	<0.001	76.1
	Own street/court/footpath	43.5	0.43	<0.001	67.4
	Nearby streets/court/footpath	19.6	0.44	0.004	78.3
	Park/playground	17.4	0.37	<0.001	71.7
	Facilities or sports ovals where your child can be active	19.6	0.37	0.001	71.7
	School grounds for free play outside school hours	0.0	a		100
	Other places where your child can be active (e.g., bush)	4.3	0.22	0.002	76.1

a Kappa statistics could not be computed.

4. Discussion 

This is the first study that we are aware of to develop and assess the reliability and validity of items examining the frequency with which children play in various outdoor locations. Overall, the reliability study indicated that the survey items displayed at least moderate reliability (11 of 16 items displayed reliability of ICC>0.50) when administered on more than one occasion to a small sample of parents. It is important to consider that low reliability may be a result of measurement error or because the behaviour that is being assessed is not very stable over time and may also vary according to other factors such as the weather. For example, there may have been actual differences in the frequency with which children played in the yard between the test and retest periods and therefore it may have been difficult for parents to report a ‘typical’ weekday.6 Interestingly, in the current study lower ICC values were obtained for items assessing play at locations where parents reported that their child played less often and it is possible that low values may have been a result of small variability between participant scores.5

The validity test results showed kappa statistics ranging from 0.22 to 0.60 and percent agreement ranging from 63% to 100% and therefore provided evidence for reasonable agreement among the two measurement instruments. Lower κ values were obtained at locations in which parents reported their child playing less often, a finding which is not atypical for items with low prevalence of responses in certain categories.7 Results from this study are limited to comparisons obtained from information recorded in the log book for only 7 days and further assessment of validity could be conducted using a longer time period. The results from this study were at least comparable to other validity studies of parental proxy-report measures of the time pre-school-aged children spend playing outdoors.2

Finally, it should be considered that both studies involved parent-report which is a subjective measure that relies on the parent accurately recalling where their child played and could involve a degree of error or self-report bias. In addition, the items that scored lower reliability or validity results may need further development or evaluation before being used in subsequent studies. For example, future research may consider using geographic information systems (GISs) to validate reports of places children use to be active and also examine sensitivity of the instrument to change over time or resulting from an intervention.

5. Conclusion 

Overall, the results suggest that the survey was generally a reliable and valid instrument for assessing the frequency with which children play in particular locations, particularly for locations where children played frequently. Evidence for the reliability and validity of items assessing children's play is novel and important considering the need to promote children's physical activity in a variety of settings.

Practical implications