Recently Added

Remote Learning Surveys – Spring of 2020 – Summary of findings

August 11, 2020

In March, I was asked by two Heads of School (one Canadian and one American) to work with their senior teams in the development of short surveys of parents on the topic of remote learning. As often happens, these conversations led to others, and customized surveys of faculty and students were also created.

As the COVID-19 crisis loomed overhead, my phone continued to ring. What followed became a very interesting exploration.

Between early April and early June, I ended up working with 48 independent schools, conducting remote learning surveys of parents, students, and/or teaching faculty. A total of 99 surveys were completed during this period. A small number of schools conducted their surveys twice. In this summary, results are included only once for each school.

Aside from the two originating schools, all other surveys were standardized, containing exactly the same questions: for parents, 8 questions plus an open comment box; for students, 9 questions plus 3 open comment boxes; and for faculty, 12 questions plus 1 open comment box.

I should say that this standardized format was an entirely new experience for me. For more than 25 years, I’ve been conducting customized comprehensive constituent surveys for independent schools, each survey including between 90-140 questions. This work involves extensive dissection of results, typically yielding more than 1,000 pages of tables and graphs for each survey, as well as my own 80 page ‘plain English’ interpretive analysis.

With a growing number of schools on the line this Spring, I had to figure out quickly what we could do to help, if anything, and how to do it without sacrificing our standard of quality. If I can’t do it right, I won’t do it at all. Very quickly I decided that, as a small group, we were in no position to develop customized surveys for a potentially large number of schools during this very tight time frame. Cross-tabbing and the creation of graphs, along with my own work on the analysis of results, for this number of surveys would have taken us well into 2021 if we’d gone the customized survey route. Schools needed top-line results ‘yesterday’ without detailed digging into the data.

The standardizing of survey content accomplished two things. First, it largely put me out of work, since my principal role in the business is the development of customized surveys and the interpretation and presentation of results. Second, it greatly expanded our capacity. With all the parent surveys identical in content, we were able to automate much of the data management side of things. Timelines were dramatically reduced from start to finish. On more than one occasion, I took a call from a Head of School, enquiring about the project, and two days later, they received results. Typically, for a comprehensive survey of parents, it takes six weeks to create, review, and revise (and review and revise, and review and revise) the questionnaire, six weeks to collect the data (with weekly badgering of parents), and four weeks to work through the analytics, placing me in a position to present results.

Unlike schools, the COVID-19 crisis gave me a sabbatical for a few months, with a number of our comprehensive surveys postponed from this past Spring to the Autumn term. So… I had time to take this project on. I like data, lots of data. The more the merrier, so I wanted a large pool of data coming out at the other end. Hoping to encourage more schools to take advantage of the opportunity, I opted to keep the decision simple, and made it available at no cost. No variations (don’t ask!)… but no cost.

Working with a small number of Heads, (special thanks to Nathan, Martha, and Larry), the surveys took shape and were made ready to launch for interested schools.

This summary reflects results from 84 of the standardized surveys (and only one per school per type of survey). This includes: 44 parent surveys; 23 faculty surveys; and 17 student surveys.

 

From the parent surveys:

Results incorporate participation by 13,200 parents, with an average of 300 per school, a high of 725 and a low of 60. The median was 273. The average rate of participation among families is 79.2%. This is somewhat higher than our typical response rate for comprehensive parent surveys (recently in the range of 73%), owing both to the length of the questionnaire, and to the urgency of the remote learning topic. The response period averaged about 36 hours, including an email from the Head of School on the launch date, and one reminder next day, stating the deadline as midnight on the second day. Results were prepared and sent to the Head of School by noon on the third day.

The table, below, portrays summary results across 44 parent surveys. To read the first row in the table, the average number of participants, across schools, was 300, with a wide range from low to high. The standard deviation (164 in this case) depicts how wide on either side of the average we have to go to include 68% of surveys.

For the second measure, agreement with the statement, “I trust the School’s leadership to make decisions that first protect the health and safety of my child and our family,” the average is 4.5, with a range from 4.3 to 4.8. From the standard deviation, we can say that just over two-thirds of all participating schools were rated for this measure in a very tight range: 4.5 +/- 0.12.

Similarly tight dispersion is revealed for all measures, with the largest spread for the final item in the table, “My child is adapting well so far to the remote learning environment.” Stressing the prohibitively small number within sub-groupings among schools, my observations tell me that ratings for this measure are lower among parents of younger students. This, in turn, may reflect greater challenges on the part of parents of younger students in adapting to remote learning. This makes intuitive sense, but would require more research to confirm.

Best Scores Report - Parent Survey on Remote Learning
Average
Low
High
Median
#
St.Dev.
# of respondents
300
60
725
273
44
164
Please indicate your level of agreement or disagreement with each of the following statements:            
1. I trust the School's leadership to make decisions that first protect the health and safety of my child and our family.
4.5
4.3
4.8
4.5
44
0.12
2. The School has been proactive in keeping me informed as the COVID-19 crisis continues to unfold.
4.5
4.1
4.8
4.5
44
0.16
3. School officials have been readily available to answer my questions related to the School's response during this period of uncertainty.
4.3
3.9
4.8
4.3
44
0.19
4. The School has demonstrated appropriate flexibility in response to a rapidly changing environment.
4.4
4.1
4.8
4.4
44
0.15
5. I support the School's move to remote learning as an interim measure.
4.5
4.1
4.8
4.5
44
0.16
6. I am comfortable with my understanding of how the remote learning program will affect my child.
3.9
3.4
4.3
3.9
44
0.19
7. My child is adapting well so far to the remote learning environment.
3.9
3.3
4.3
3.9
44
0.24
8. I am confident that the School will continue to offer a fulfilling educational program for my child.
4.0
3.6
4.4
4.1
44
0.19

I should declare the potential limitations of this work:

While these ratings all appear strong, it is impossible to say how they would appear in comparison to those from public school parents. All schools participating in this project are independent schools. This said, having conducted more than 450 comprehensive attitudinal surveys for independent schools before this project (174 of them parent surveys), these results look strong to me.

The number of schools within the limiting distinctions described above is too small to break them apart into meaningful cells without abandoning any sense of statistical validity (so, please don’t ask). This said, there was one comparison I was hoping to make, and have been hoping to make for the past 25 years. With the standardized format, I was finally able to make a same-base comparison in attitudes between Canadian parents and American parents. I have been saying through the years that parents are parents are parents, but Heads of School always meet this with a raised eyebrow. In the end, here’s what I found.

Of 44 schools included, 27 are Canadian and 17 American. The average number of parent participants for each survey: Canadian – 302; American – 296.

In comparing the average results across Canadian schools with those across American schools: Not even one of the eight questions revealed differences of more than 0.075 on the rating scale from 1.0 to 5.0. The average of the eight scores for Canadian schools was 4.256 and for American schools, 4.250.

Moreover, for agreement with the statement, “I support the School’s move to remote learning as an interim measure” ratings differed by just 0.0005, and this only by rounding.

So… parents are parents are parents. Nice to know. We’re more the same on either side of this border than we are different. Let’s not forget that this is the longest undefended border in the world… current circumstances not considered.

We have also prepared a table to include answers for all parents across all 44 schools. With 13,200 participants, there is a very strong basis for correlation analysis between answers. The strongest correlation was revealed for the two measures: “I trust the School’s leadership…” and “The School has been proactive in keeping me informed…” at 0.78. There was also strong correlation between “The School has been proactive in keeping me informed…” and “The School has demonstrated appropriate flexibility…”. So… of no surprise, communications, and flexibility in response during uncertain times are key correlates to trust in leadership.

 

From the student surveys:

Student results from 17 schools are shown in the table, below. Points worth noting include:

Best Scores Report - Student Survey on Remote Learning
Average
Low
High
Median
#
St.Dev.
# of respondents
310
63
935
300
17
202
A. How easy is it for you to access the information you need for your classes each day?            
 
not very easy
4.8%
0.8%
9.3%
4.2%
17
2.73%
 
somewhat easy
45.5%
34.4%
54.9%
46.8%
17
6.03%
 
very easy
49.6%
39.7%
63.5%
49.2%
17
7.72%
B. How would you describe your daily class schedule?            
 
not very manageable
5.3%
1.6%
12.0%
4.8%
17
3.41%
 
somewhat manageable
46.9%
34.1%
58.9%
47.6%
17
7.03%
 
very manageable
47.8%
34.5%
61.6%
47.6%
17
8.63%
C. How would you describe your school workload, including homework?            
 
not very manageable
24.0%
10.2%
45.8%
24.8%
17
7.77%
 
somewhat manageable
57.4%
42.1%
68.6%
58.2%
17
6.86%
 
very manageable
18.5%
9.7%
26.4%
18.7%
17
5.33%
D. How would you describe the amount of class time you have with teachers?            
 
not enough
10.3%
1.3%
28.5%
7.4%
17
8.57%
 
about right
75.0%
49.2%
88.6%
78.7%
17
11.19%
 
too much
14.7%
3.0%
49.0%
9.3%
17
13.08%
E. How would you describe the availability of your teachers for extra help?            
 
not adequate
3.7%
0.7%
14.6%
2.1%
17
3.58%
 
somewhat adequate
19.3%
9.5%
36.0%
18.2%
17
6.39%
 
quite adequate
45.2%
30.7%
59.5%
45.2%
17
8.04%
 
I have not had the need.
31.8%
18.0%
44.3%
32.0%
17
8.89%
F. How much "education-related" screen time (including classes, homework, and collaboration with peers) do you spend on a typical school day?
4.8 hours
3.5 hours
5.6 hours
4.9 hours
17
0.49
G. How would you compare remote learning to in-person learning as an environment for you to be an effective learner?            
 
remote learning is much more effective
3.5%
0.8%
7.8%
3.2%
17
2.02%
 
remote learning is somewhat more effective
6.2%
2.9%
11.4%
5.7%
17
2.28%
 
they are about the same
10.7%
5.6%
16.1%
11.3%
17
2.92%
 
in-person learning is somewhat more effective
27.6%
19.3%
37.2%
26.9%
17
4.82%
 
in-person learning is much more effective
52.1%
37.2%
67.4%
55.2%
17
9.33%
H. How would you describe your experience in maintaining a sense of community and connectedness to your classmates and teachers?            
 
not at all successful
19.3%
9.9%
30.0%
17.7%
17
6.64%
 
somewhat successful
64.7%
58.0%
72.6%
64.8%
17
4.45%
 
very successful
16.0%
8.7%
26.4%
16.0%
17
4.97%
I. Using the scale from 1 to 10, below, please rate your sense of social/emotional well-being during this time of remote learning?
5.9
5.2
6.7
5.8
17
0.43

 

From the faculty surveys:

Results from 1,219 teachers at 23 schools are shown in the table, below. The average rate of participation was 90.8%, with a standard deviation of 9.1%. Answer formatting took a mixed approach for faculty, with some 5-point ratings and some descriptor selection ratings.

Best Scores Report - Faculty Survey on Remote Learning
Average
Low
High
Median
#
St.Dev.
# of respondents
53
12
155
43
23
32
A. Please indicate your level of agreement or disagreement with each of the following statements about your remote learning experience:            
1. Communications from administrators about faculty expectations, generally, have been clear.
3.9
3.2
4.5
4.0
23
0.36
2. Expectations for faculty, generally, have been reasonable.
4.0
3.5
4.4
4.0
23
0.25
3. I feel well supported to manage the technology side of remote learning.
4.1
3.5
4.5
4.2
23
0.25
4. In general, I am able to balance working from home with my family commitments.
3.7
3.2
4.4
3.7
23
0.28
5. I feel well supported by the School's leadership in this time of remote learning.
4.0
3.2
4.5
4.1
23
0.35
6. I feel successful in maintaining a sense of connection with my students.
3.6
3.0
4.0
3.7
23
0.25
7. I feel successful in helping my students to remain connected with each other.
3.4
2.8
3.7
3.4
23
0.23
8. I feel successful in maintaining a sense of collegiality with my colleagues.
3.6
3.0
4.1
3.5
23
0.29
B. How would you compare remote learning to in-person learning as an environment for you to be an effective teacher?            
 
remote learning is much more effective
0.5%
0.0%
2.5%
0.0%
23
0.84%
 
remote learning is somewhat more effective
2.4%
0.0%
8.1%
1.8%
23
2.47%
 
they are about the same
5.4%
0.0%
13.5%
4.9%
23
3.81%
 
in-person learning is somewhat more effective
26.7%
11.9%
53.8%
25.0%
23
9.69%
 
in-person learning is much more effective
65.1%
34.6%
81.4%
65.7%
23
10.08%
C. Which of the following might be most helpful in supporting you as a remote learning teacher going forward?            
1. longer block periods
9.7%
0.0%
46.7%
5.7%
23
11.49%
2. additional class periods per cycle
17.1%
0.0%
49.2%
11.9%
23
13.45%
3. increased independent activity for students
31.3%
8.3%
50.0%
32.6%
23
12.16%
4. increased "office" hours of accessibility to students
13.9%
2.4%
35.3%
12.1%
23
8.62%
5. decreased "office" hours of accessibility to students
5.1%
0.0%
17.1%
3.6%
23
4.67%
6. not making any changes to the schedule
46.7%
27.3%
75.5%
45.5%
23
12.64%
D. Thinking about the essential curricular and instructional priorities of your class, which of the following online teaching skills would it be helpful for you to receive support in at this time?            
1. designing discussions for remote learning
19.4%
0.0%
31.7%
20.7%
23
7.56%
2. designing group activities for remote learning
45.7%
30.9%
73.3%
41.7%
23
10.04%
3. transferring an in-class exam to an online version
18.9%
0.0%
39.0%
20.0%
23
11.16%
4. designing alternate assessments in a remote learning environment
40.8%
23.7%
59.2%
41.5%
23
10.41%
5. designing self-guided lessons
24.9%
14.3%
42.9%
25.0%
23
7.66%
6. designing projects for remote learning
29.3%
12.2%
53.3%
27.0%
23
10.11%
7. designing student-conducted experiments for remote learning
11.6%
3.8%
16.7%
11.1%
23
3.33%
8. classroom management in online learning
13.9%
5.4%
20.4%
15.2%
23
4.36%
9. promoting community in a remote learning environment
42.1%
22.7%
59.3%
42.9%
23
10.84%
E. Using the scale from 1 to 10, below, please rate your sense of social/emotional well-being during this time of remote learning?
6.3
5.6
8.0
6.3
23
0.54

 

This was certainly an engaging project, and as I often say, “Engagement is Everything”. Thanks to all the participating schools. I’m grateful for the learning opportunity, and the meaningful activity with which to keep myself busy. I hope it was a worthy experience for you as we continue to explore our meandering journey through these uncertain times.

Life is a journey. Life is good.

With respect,

Kevin Graham

I welcome your feedback. Feel free to contact me by e-mail.

To help me avoid receiving a ton of spam, I’ll ask that you please replace the parenthetic content, and the parentheses, of course, with the @ sign. Thanks.

kevin(at sign)lookoutmanagement.com