Reflection on Assignment 1
As I reflect on the process, it does seem we’ve come a long way since we first started. In this blog I will share the following:
- Sections I was responsible for in completing Assignment 1
- Working in a team
- Giving feedback to other groups’ evaluation plans
Below I have itemized the sections I was responsible for in the planning process and how I went about completing it.
1. Background and rationale – I deliberately gave as much information as I could about the design and development of the iLessonPlan (iLP) as it is a prototype lesson planning tool still in development. Initially it was a concept in my head, then it took shape and became a concrete object as in the tool itself. It is still in the alpha stage of development so it is challenging for people on the outside to fully understand what it is. So that was the reason for including as much information as I could about it in this section.
I do appreciate Bronwyn’s comments to keep the background directly related to the evaluation because if you don’t then you could lose your reader … so I trimmed the Background section and have included the relevant parts in the Appendices.
2. Process of using the iLP – the same argument as above applies. Tom and Mark gave very useful feedback as they thought it better to list the steps. I thought this was a good idea as previously I had described the steps using paragraphs. Again the rationale was to make it reader friendly. Then Tom made it even better by including screenshots for each stage in the process. I’m very pleased at the end result as ‘a picture says a thousand words’.
3. Aims and Evaluation Questions – If I remember correctly, either Tom or Mark or both had started off this section, and then I added my part.
I knew that usability testing was an important part of the formative evaluation as the iLP is a tool which had not been trialed formally before. Reading the e-Learning Guidelines (Massey, 2007) provided valuable insight as to how the aims could be structured. I felt quite excited as I was reading the Guidelines as I thought how brilliant to have a set of user friendly guidelines to use.
We started with something general and have now refined the aims and sub questions further. Now there are two aims linked to two e-Learning Guidelines as below:
a. ST9: Does the iLP help ESOL trainee teachers to successfully develop their knowledge of lesson planning?
b. MO1: Can the ESOL trainee teachers easily use the iLP?
Earlier I had included another aim and e-Learning Guideline as below:
SD3 Do ESOL trainee teachers gain knowledge relevant to planning an ESOL lesson?
I felt this was a relevant aim but due to pragmatic reasons we decided to stick to the two above. I might add this at much later phase of evaluating the iLP… possibly evaluating it next year after it has been hosted on a website.
Table 1: An overview of the essential elements in a formative evaluation plan – I find working with tables one of the best ways to relate information logically and coherently. So I often use them to do just that before I start writing the report. Also using tables helps me to easily identify easily missing information and information that doesn’t quite fit in the boxes. So I was quite pleased to find that Tom, Mark and Bronwyn liked Table 1.
4. Methodology – It was interesting to read up on the different evaluation models. Both quantitative and qualitative data collection methods were included though each does have its limitations. But using both together provides richer data which then provide stronger evidence for the results and conclusion.
I must remember to note things which work and do not work while collecting the data. Also to note how emerging problems were resolved and if they couldn’t be resolved then to account for them in the interpretation of the findings. Hmm… the next few weeks will be very challenging for sure…
5. Sample and Instrumentation – Again this section was started off by Tom and/or Mark and I added my contributions. Due to the discontinuation of the CATESOL Programme, thetarget population for whom the iLP was developed for was unavailable so we had to resort to using trained ESOL teachers. A real challenge has been to highlight this point from the beginning of the evaluation plan. Mark and I did that and I tidied it up further to be more consistent.
6. Conclusion – On hindsight I should have included in the design and development of the iLP, an evaluation component as it is an important part of the design and development of the tool. The ideas which informed the design and development were conceived with one product/outcome/output in mind but by incorporating evaluation many of the issues could have been resolved much earlier. But then that is part of the creative process isn’t it? Learning from one’s mistakes and making sure that it doesn’t happen in future. That to me is what learning is all about. Evaluating is a cyclical process and I do expect to re-evaluate the iLP at a later stage. To me this is an essential part of refining the tool.
B. Working in a team
As Fred said in his blog, there are advantages and disadvantages working in a group. A great advantage is the multiple perspectives brought to the project and the collaboration which enhances the quality of the work put in. Each member contributes differently and to me it is important to ensure each member is valued and appreciated in relation to the knowledge, experience, and skills brought to the task.
Communication is crucial to keep everyone in the loop and updated on what’s happening. Working on the wiki initially was a nightmare. We resolved this problem by having weekly f2f meetings, emailing constantly each other our contributions, versions of the plan etc using attachments and posting the updated versions on the wiki so that Bronwyn had an idea of the collaboration taking place behind the wiki.
Emailing also has its challenges as the tone, language and message communicated may not necessarily be the one intended by the writer. And how it is received and perceived by the reader is another matter altogether. One option would be to use Skype as this synchronous form of communication would help resolve some of the problems faced in asynchronous communication. But as I found out, MIT does not allow lecturers to have Skype installed for professional development (PD) purposes. So on one hand we are encouraged to do more PD but then are not allowed to have the tools, in this case Skype, to do so.
On a personal note, I’m someone who constantly reflects on how things can be improved and further progress made. So I do tend to take it upon myself to take the necessary steps needed to initiate changes. I must remember that I’m part of a team and therefore consultation is crucial before action is taken.
Tom and Mark are excellent team members and I’m happy to be working with them on this assignment.
C. Feedback provided to other teams’ evaluation plans
b. Sandra H and Rosanne