Four distinct cases are discussed when determining effectiveness of instruction in mastery vs. nonmastery using pretests and posttests.
- Student Nonmastery = Student Mastery
- Student Nonmastery = Student Nonmastery
- Student Mastery = Student Mastery
- Student Mastery = Student Nonmastery
After discussing the pretests and posttests, compiling of the prototype is discussed. The general consent is that enough information should be provided to effectively test usability. Additionally, several "strands" will be included, specifically the the strand that delves the deepest. Using codes to represent links, and including these codes on tabs within the paper prototype is also discussed.
The actual testing of the paper prototype relies on the evaluator, to provide the volunteer with the paper prototype. The evaluator will document the volunteer's paths, comments while working, and attitude toward the prototype. A pilot session is strongly encourages, to work out the kinks of the entire process. Evaluators should be aware of techniques regarding noncommittal responses, prompting, and probing. These techniques commonly prove beneficial when gathering information during the prototype testing phase. After the testing phase is complete, the evaluator will go through 5 steps.
- Assemble Data
- Identify Patterns
- Define Problems
- Diagnose Design Problems
- Prioritize Revisions.
Thoughts
Prior to this reading, I was extremely critical of a paper prototype. How effective can an out-of-context website be? In my view it is like a book that has been adapted to a screen production, there is always information left out. However, after reading these chapters, I began to understand the benefits of a paper prototype. Common user difficulties and other instructional errors may be noticed before continuing to the actual design phase of the website. A paper prototype may save a lot of time and effort when properly used.
Using a notebook or 3 ring binder to create your rough draft seems feasible and realistic. I am still reserved in the idea that a paper prototype would be needed to discover linking errors and such. I think the actual links and other items should be followed on the computer and tested at great length, but I'm also paranoid about dead-end links. The information I gathered through this reading will be put to use for our paper prototype. The use that I saw for the paper prototype that most directly impacts me is simply for the planning stage of a website. It would be beneficial to have a visual layout and corresponding information for a site before I begin developing it. I am an individual who typically writes a paper before my outline, so I typically develop a mock-up for my own visualization rather than try to break it down into pieces. I suppose I am a "big picture" person.
~Mikah
Resources
Frick & Boling (2011). Effective web instruction: Handbook for an inquiry-based process. Indiana University.
I would have to agree, I am not sure that I like paper prototypes still. I think you could do essentially the same thing on the computer. Maybe in powerpoint. The tester can visually point to it and then you can take him to the page he intended to do.
ReplyDeleteI could maybe understand like you said big picture to be drawn. Like developing a website you need this content area, etc. But I just do not think the fact that it is paper is all that is necessary. I think it should be showcasing more of the idea to prototype.
Re: bias for or against paper prototypes.
ReplyDeleteAs with any of the processes and tools we have available as designers, if we're smart we apply the right tool for the right job. I can see value in the paper prototype process when the project has certain parameters but not, at least in my experience as an ID so far, for every project. For example, prototyping would be appropriate for large scale projects, big budget projects, large design team projects, critical-to-the-client projects.
The reality is that small budget, limited scope, single designer projects may benefit just as much from informal, small sample testing. That said, I would attempt to build the paper prototyping step into any project I could get it approved for, just for the experience. And not only for myself and/or the design team. The client is the biggest stakeholder in the success of the project and the prototyping step can go a long way toward gathering hard data to make your case for learner-centered design.
I feel like I'm on the fence, rather than taking a hard stance either way, but I can see the pros and cons of both sides.
ReplyDeleteBrittany, that is a very valid point of using PowerPoint to essentially create the "paper prototype". It would be the same concept, but allow for a different set of visuals. The visuals would be in the format of computer/web rather than on paper which could prove beneficial.
MediaSage, I can also see how a paper prototype would be beneficial for the stakeholders. Also, it would increase my experience in working and prototyping as an ID. While the entire process is clear and logical. I can see subjects going through the paper prototype phase seeing it as silly, thus not taking the process seriously and scewing the results.
~Mikah
Hi Mikah - great post and reminder of the 4 cases of pre/post-testing. One element I found missing in the observation phase was sharing findings with the subject and getting confirmation. This is different from a reactionnaire and a post-test. This involves going over your notes and sharing some or all of your observations. I think it's an integral step that the observer should take in order to confirm findings. A great deal of the recording of data is subjective - especially Frick & Boling's advice to note self-deprecating comments. It's important to not assume too much or read extensively into subjects' thoughts without confirming our assumptions.
ReplyDeleteI have made this mistake more than once while doing peer and evaluative observations of teachers. Because I've come into a class as an outsider with much less context and background on the learners, the environment, the class 'm.o.', I have assumed certain dynamics and interaction patterns that were inaccurate. I've found that, especially in evaluative observations, it's important to confirm findings by sharing them and asking for the subjects' perspectives.
I agree, the paper protoype is no longer useful because creating a Web site shell can be so easy. When considering a high school or college-age audience for the instruction, the paper prototype will make the developers seem out of touch with their needs. I believe you will get better feedback from on online, web-based prototype from this audience.
ReplyDeleteIf you are creating a paper prototype because the clients need to see it, then this matter is different. The paper prototype would need to be more polished for this audience, if they are paying you to create instruction for them.
Either way, I believe a computer-based prototype is what will get the best response now. (Remember, the article was created in 2004, designing is so much easier now...)
Michele