Week 25 Collecting Data and Evidence

Why collect data and evidence?

The Ministry of Education (n.d) points out “without evidence to back them up, hunches remain subjective.” The Ministry goes further, saying that well-planned data gathering can “reveal new patterns of insight, justify change and motivate colleagues, stakeholders, and school leaders into taking action.” (Ministry of Education, n.d).

In criterion 3 of Research assessment 2, as evidence of community engagement, we are required to include in your teacher inquiry plan how you plan to collect data from the relevant members of your community.

We plan to include both qualitative data from student survey forms and quantitative data from written assessments e-asttle measurements of written samples.

Using mixed methods (often a combination of quantitative and qualitative methods) will give richer data, hence our choice on a combination.

The choice of using Term 1 and Term 2 data from written samples means we do not need to re-collect data or re-analyse information that has already been collected and therefore reducing workload.

Survey/questionnaire

A survey or questionnaire is used to obtain opinions from respondents to provide information through a set of questions (Babione, 2015).

A strength of this approach is that a survey or questionnaire can be distributed and collected quite easily (Babione, 2015), especially since there are a number of online survey tools such as Google Forms and Survey Monkey which make it fairly straightforward to create, send and collect responses. A survey can be done with a small or large number of respondents and can include both quantitative and qualitative data. Some online survey tools also include some automatic data analysis features.

The downside can be that you are not always able to control the number of responses, especially for an anonymous survey, and the responses can be biased or the responses may not always reflect the real opinion of the respondents.

Now, consider the pros and cons of doing a survey and decide if you would like to use surveys as one of your data collection methods. If you wish, read  the section about surveys extracted from Efron and Ravid’s (2013) book (supplementary). The extract provides guides to help you write survey items with good examples. Additionally this checklist for developing a survey (Peters & Irish, n.d.) (supplementary) provides a list of questions to help you check your survey design. Remember to include the tool (the survey questions) in your assessment submission if a survey is one of your chosen data collection methods.

If you plan to conduct a large scale survey, for example, across the whole syndicate or the whole school, you can use the calculator on this web page (http://www.surveysystem.com/sscalc.htm) to determine the appropriate sample size for a survey (e.g. how many responses you should collect) to have the result within a certain margin of error (also called the Confidence Interval) and for a given population (i.e. the total number of students in your school).

Interviews:

Interviews are frequently used to obtain verbal perspectives and opinions from individuals or group about their understandings and experiences, thoughts, feelings, motivations, and the actions of others (Babione, 2015,p.124).Here is an example of an interview schedule including the purpose of the interview and interview questions.

A strength of interviews is that they provides a richer and deeper perspective about the situation and help to verify information from other sources.

A constraint of interviews is that it can take time to arrange and conduct the interview session and analyse the collected data. Because an interview is person-to-person interaction, interviewees might answer to please the interviewer instead of telling them what they really think and feel, especially if the interviewees are afraid of unfavourable consequences as a result of the interview.

If you choose to interview as your data collection tool you would like to use in your teacher inquiry plan, read this section about interviews from Efron and Ravid (2013)’s book (supplementary). The section outlines different types of interview, the interview process and how to develop the interview questions.Remember to include the tool (the interview questions) in your assessment submission an interview is one of your chosen data collection methods.

Observations

Efron and Ravid (2013) define observation in research as the act of purposefully and systematically observing the activities, people, and physical aspects of the educational setting.

The strength of observations is, as Babione (2015) points out, that observing humans in natural settings provides insight into the complexity of human behavior and interrelationships among groups. Observations allow you to collect data such as nonverbal behaviours, gestures, and body language, the dynamics of the setting that cannot be obtained through interviews or surveys.

If you choose observation as a data collection tool you would like to use in your teacher inquiry plan, read this this section about observation extracted from Efron and Ravid (2013). The extract explains the different types of observation, its process and how to develop an observation form such as a protocol form for qualitative observation and behaviour log, tally sheet and checklist for quantitative observation. Remember to include the tool (the observation protocol form/behaviour log/tally sheet/ checklist, etc) in your assessment submission if observation is one of your chosen data collection methods.

Artifacts and documents

Artifacts and documents are records that provide contextual information for a study (Babione, 2015). Examples of artifacts and documents include student work, school reports, demographic information, grades, test scores, portfolios or teacher journals.

The advantage of this data collection method is that the artifacts and documents may already be available during the course of your teaching and you don’t have to design an extra tool to collect the data.

If you choose artifacts and documents as the data collection tool you would like to use in your teacher inquiry plan, read this section about artifacts and documents extracted from Efron and Ravid (2013). The extract provides a checklist of suggestions and guidelines for gathering and using official and personal artifacts and documents.

Pre and post intervention data

This is part of what we plan to use. Pre-test data from Term 2 data and post test data in Term 3

What baseline data do you need to collect? Baseline data is data collected at the start of an inquiry project that allows you establish the current situation which will enable a point of comparison for data you collect throughout your inquiry.

The video on developing a quantitative research plan (supplementary) is relevant as we are including experimental methods in our data gathering (e.g. comparing pre-test and post-test student results).This will be used as we are developing an intervention designed specifically to raise student achievement in writing and skilled communication. We will give them a post test after the intervention in term 3.

For each data collection method you need to explain why you have chosen it, the data it will provide for you and why this data will be important.

Evidence of engagement with relevant community members to inform the inquiry plan

As part of the assessment 2 criterion 4 you need to show how evidence of engagement with relevant community members (e.g. colleagues or stakeholders) informed the inquiry plan or part of the plan. As discussed in week 24 class notes, the relevant communities for your Teacher Inquiry Plan can be your students, colleagues, whanau or school leadership. You need to seek engagement from these community members in the form of EITHER the data you gather from your data collection tools if you can OR the feedback about your teacher inquiry plan such as the plan milestones, the data collection tools, the ethical considerations, etc.

You need to record the engagement to show as evidence. It can be either a presentation of the data you collected from your data collection tools (if available) or written feedback (e.g. email correspondence, G+ reply posts, notes you made during a conversation with your community members) or a video recording of the feedback.

Additionally, Assessment 2 criterion 4 requires that you reflect how the engagement with the community has informed the inquiry plan or part of the plan. You need to explain what changes you made to your plan after considering the data or the feedback (e.g. did you change your research question or data collection tool(s)?).

References

Babione, C. (2015). Practitioner teacher inquiry and research. USA: John Wiley & Sons. (e-copy available in Unitec library).

Easton, C. (2012, April). Data for self-evaluation. SecEd. NFER. Retrieved from http://www.nfer.ac.uk/schools/data-for-self-evaluation-seced.pdf

Efron, S. E., & Ravid, R. (2013). Action research in education: A practical guide. New York, NY: The Guilford Press. (e-copy available in Unitec library).

Ministry of Education.(n.d.). Data analysis. Retrieved from http://elearning.tki.org.nz/Teaching/Teaching-as-inquiry/Data-analysis#js-tabcontainer-1-tab-2

Riel, M.( 2014, 28 April).T7: Data collection.[Video file]. Retrieved from https://www.youtube.com/watch?v=PyILgB1R5Cc

Shaddock, A (2013). Using data to improve learning : A practical guide for busy teachers. ACER Press.