Assignment #5 - Design and Test a Survey

ECUR 809

Assignment #5 – Design and Test a Short Survey

The survey that I created is intended for the teachers who teach in the Grade Nine Learning Community at my school. It was difficult to create because there will undeniably be bias in their responses; I considered creating one for grade nine students regarding the program, but came to the conclusion that that could be entering a sensitive situation. Most of the teachers in the program have helped to create it, so they would see the changes and the impact on the students more than any other teachers. The new teachers in the program had no knowledge of what the situation was like before the program, but would still have valuable input about it.

My first survey was 7 questions with a mix of rated responses and longer answers. I sent the survey to four people for feedback. I received feedback from all four people, and changed my survey accordingly. I still feel like it could be better – the bias concerns me, although the feedback can still be valuable.

This is the first survey (done and sent through Survey Monkey):

Use link, or see below: http://www.surveymonkey.com/s/KGBC9GJ

Grade Nine Learning Community Survey

 

1. How long have you been teaching in the Grade Nine Program?

2. Do you believe that the Grade Nine Learning Community has improved attendance compared to before the program was implemented?

No, not at all	Somewhat	Yes, there has been improvement	Yes, significantly

3. Do you feel that the Grade Nine Learning Community helps to build relationships among students more so than without the program?

No, it does not	Somewhat	Yes, for the most part	Yes, significantly

4. Do you find that teaching a variety of subjects instead of one specialized subject is not utilizing your strengths?

Yes	Somewhat	I have learned to adapt	No, it is working successfully

5. Do you believe that the Grade Nine Learning Community has long term benefits?

No	Somewhat	Yes, for the most part	Yes, definitely

6. What do you believe are the greatest strengths of the Grade Nine Learning Community?

7. What do you believe are the disadvantages to Grade Nine Learning Community?

The feedback I received was valuable; I had a feeling that my questions were vague or too open ended but was not sure how to fix them. Here are my revisions:

I added a page at the beginning – the feedback given stated that a purpose for the survey would be beneficial, so I stated who and what the survey is for.

#1 – Stayed the same

#2 – My original question started off with do you believe, and all the feedback stated that belief should not be part of the question – they should rely on stats and evidence. So I changed the question to make it less about opinion.

#3 – I kept this the same, but then added a question after it about relationships with teachers – one person suggested that might be just as relevant in this program.

#4-Kept the same – no feedback was given about it.

#5-I received a number of suggestions on how to improve this question – it was much too general and vague, especially to answer in a rating scale format.  Originally, I asked if the program had long term benefits. The feedback was to be more specific; people were unsure how long term I meant. Benefits was also quite vague. My new question is #6 and asks whether the program benefits students in their later years of high school. I then added one more question about resiliency, as that is something that affects the students long term. I made this question a response based answer, and I felt that the information would be valuable to determine how much benefit the program actually has for students.

#8 and #9 were kept the same, as there was no feedback on these.

My revised survey:

Use link, or see below: http://www.surveymonkey.com/s/KWCCQ5X

The purpose of this survey is for grade nine teachers to reflect on the impact of the Grade Nine Learning Community, and establish if the benefits are worth keeping the program.

Grade Nine Learning Community Survey Revised

 

1. How long have you been teaching in the Grade Nine Program?

2. Has the Grade Nine Learning Community improved attendance compared to before the program was implemented?

No, not at all	Somewhat	Yes, there has been improvement	Yes, significantly

3. Do you feel that the Grade Nine Learning Community helps to build relationships among students more so than without the program?

No, it does not	Somewhat	Yes, for the most part	Yes, significantly

4. Do you feel the Grade Nine Learning Community helps to build lasting relationships between teachers and students more so than without the program?

No, not at all	Somewhat	Yes, for the most part	Yes, significantly

5. Do you find that teaching a variety of subjects instead of one specialized subject is not utilizing your strengths?

Yes	Somewhat	I have learned to adapt	No, it is working successfully

6. Does the Grade Nine Learning Community benefit students in their later years in high school?

No	Somewhat	Yes, for the most part	Yes, significantly

7. How has the Grade Nine Learning Community affected resiliency among students?

8. What do you believe are the greatest strengths of the Grade Nine Learning Community?

9. What do you believe are the disadvantages to Grade Nine Learning Community?

Assignment #4 - Logic Model

GOAL	OBJECTIVES	ACTIVITIES	OUTPUTS	OUTCOMES	IMPACTS
Statement of the overall purpose of the project	Specific statements of what the project sets out to accomplish	Specific tasks to complete through implementation of the project	Immediate results (direct products of project activities)	Intermediate results (1 to 3 years after project starts)	Long-term results (3 to 10 years after project starts)
To increase achievement  rates in grade 9 students To improve attendance in grade 9 students. To build resiliency in grade 9 students by establishing relationships with one main teacher.	Students will attend more regularly because of one teacher tracking them.( 3 year pilot) Students’ achievement will increase based on improved attendance (3 year pilot) Students will understand the value of attendance and success and take these skills with them to grade 10 when they are not in specific groupings. (1 year) Students will feel a sense of responsibility based on relationships established – between students, teachers, and parents.	Establish the program as 3 year pilot project Identify 5 teachers who are willing to teach all subjects to one group of grade 9 students all day Organize preliminary reading tests (Gray Silent Reading Test (GSRT)) – to administer in fall Organize math assessments – to administer in fall Retests in spring Create portfolios and evaluation plans for each student Train all teachers in Assessment for Learning techniques (AFL) with Anne Davies Track attendance over the 3 year period and compare to attendance for grade 9 students in the past.	Attendance will improve in all classes. Test scores will improve Homework will be regularly completed. Attitudes towards school will improve Students will feel more of a sense of accountability because there is one teacher who is keeping track of them.	Students will stay enrolled in school. Students will realize more success in grades 10, 11, and 12 Students will set a goal to graduate Students will establish relationships with peers and teachers that will assist in their future success.	Students will graduate Students will seek post-secondary education because they see the value of education Students will be contributing members of society
Goals and Objectives represent the Strategic Directions of your Project		Activities comprise the Project Work Plan, which should include details for each activity (Who?, What?, When?, Where? How?)	Outputs link directly to project activities; activities are what is done…outputs are the expected results of what is done	Outcomes relate to your objectives; objectives are desired, outcomes are the expected results	Impacts relate to goals; a goal is desired, impacts are the expected end-results

This logic model outlines information that I have gathered about the program so far. The information mainly came from talking to teachers who have been involved in the process since the initial start up. While this was originally set up as a three year pilot project by the Saskatoon Public School Division, the initial data proved that it was highly successful in retaining kids; consequently, the division cancelled the pilot status and allowed it to continue without parameters from the division. Data was tracked the first year, and the results were encouraging; before the program, failure rates (failing at least on class) were at 50% before the program. Once the program was implemented, this number decreased dramatically. Test scores were also compared from fall and spring and results were positive. There has been no data tracked since the first couple of years, and there is no data to indicate whether the program’s benefits carried on to the following years. There is a concern that while the program shelters the grade 9 students and is overall quite positive for them, the benefits are not carrying through to the remaining years in high school. There is data collected over a four year period, but nothing was done with it, so I do have resources to work with. Teachers in the program and past administration involved in the process are also a wealth of knowledge and are willing to provide any information needed.

Assignment #3 - Evaluation Assessment

Engage Stakeholders

Who should be involved?

Grade 9 Program teachers, resource teacher, students, parents, school board

How might they be engaged?

By determining if the goals are being met and if the program is more beneficial than detrimental.

          Focus the Evaluation

What are you going to evaluate?  Describe program (logic model).

The Grade Nine Learning Community at Mount Royal Collegiate

What is the purpose of the evaluation?

To determine whether the program is allowing for a smooth transition from grade eight, and helping to retain students.

Who will use the evaluation? How will they use it?

Who/users	How will they use the information?
Program Teachers	They can make changes to the program depending on the results.
School Administrators	Determine whether the program should continue or not.

What questions will the evaluation seek to answer?

Does the program actually help to transition kids from grade eight to grade nine?

How does it prepare them for grade ten if the program is similar to grade eight groupings?

Does it help retain students more effectively than without the program? Are the relationships authentic – do they continue past grade 9 and consequently lead to success in later years.

What information do you need to answer the questions?

What I wish to know	Indicators – How will I know it?
Attendance data before and after the program was in place	If attendance improved, then the program has some benefit.
Reading and math scores before and after program	If better after, benefit to the program.
Failure rates before and after program.	If students are not failing as many classes as before the program was implemented, then it would indicate merit.
Grade 9 attendance data compared to grade 10 attendance data	If attendance is good in grade 9 and changes significantly in grade 10 it would indicate several things about the program.

When is the evaluation needed?

By the end of the 2012-2013 school year to determine if changes need to be made for next year.

What evaluation design will you use?

A mix of Stake’s Countenance Model, and Scriven’s Summative Model

          Collect the information

What sources of information will you use?

Existing information:	Attendance reports, test results
People:	Teachers in the program, resource teacher, administration,
Pictorial records and observations:	None

What data collection method(s) will you use?

*	Survey		Document review
*	Interview	*	Testimonials
	Observation		Expert panel
	Group techniques		Simulated problems or situations
	Case study		Journal, log, diary
*	Tests		Unobtrusive measures
	Photos, videos	Attendance Reports	Other (list)

Instrumentation: What is needed to record the information?

A program to compile the attendance reports to determine if change has occurred.

When will you collect data for each method you’ve chosen?

Method	Before program	During program	Immediately after	Later
Interview/survey past students of the program		*
Interview/survey teachers in the program		*
Attendance reports		*	*

Will a sample be used? 

No
Yes	*	If yes, describe the procedure you will use.
Students in grades 9, 10 and 12 will be used to gain information about their take on the program.

          Analyze and Interpret

How will the data be analyzed?

Data analysis methods:

Analyze the surveys to determine if students find benefit to the program –interview grade 9, 10 and 12 students – students who are in the program, recently out of the program, and students who have had time to reflect on the program. Teacher responses will also be analyzed to determine how they believe the program is running. Analyze grade 9 attendance and test results and compare them to the same students in grade ten

Who responsible:

Me

How will the information be interpreted—by whom?

I will interpret the information by compiling answers and looking for consistency

What did you learn? What are the limitations?

Some of the responses could be subjective, from both the teachers in the program, and from the students. There are many variables that can determine a response; grade 9 students may not be the most reflective when it comes to their learning and motivations.

Teachers of the program may be biased because they started it and are so close to it.

               Use the Information

How will the evaluation be communicated and shared?

To whom	When/where/how to present
Teachers of the program	Present reports on data and how students perceive the program and what is happening in reality.
Administration	Present reports on data and how students perceive the program and what is happening in reality.

Next steps?

Show this to the teachers of the Grade 9 Learning Community to determine where the gaps are

          Manage the evaluation                 Standards

Reports will be anonymous	Human subject’s protection		Utility
	Management chart		Feasibility
	Timeline		Propriety
*	Responsibilities		Accuracy
None Needed	Budget