RepeatedlyAnsweringQuestionsThatElicit.pdf

Repeatedly Answering Questions That ElicitInquiry-Based Thinking Improves Writing

Phanikiran Radhakrishnan, Ulrich Schimmack, and Dianne Lam

Participants engaged in inquiry by practicing how to answer questions about ajoumal article. Inquiry improves writing by helping one leam more about the topicat hand. Practice improves performance only if leamers know how to perform atask accurately (Ericsson, Krampe, & Tesch-Romer, 1993) and answering questionstrains them on the best way to engage in inquiry. We implemented an inquiry-basedwriting intervention to show that students who repeatedly answered questions wrotebetter after the intervention than before and when compared to those who did notparticipate in the intervention.

"Writing today is not a frill for the few,but an essential skill for the many" (The Na-tional Commission on Writing, 2003). Howshould teachers help students significantlydevelop this skill? Our paper identifies keyfeatures that writing activities should havewith a field study that was part of an initia-tive sponsored by a psychology departmentto improve student writing.

Teaching students how to leam improveshow they write (see Graham & Perin, 2007;Hillocks, 1986 formeta-analyses). Learning,or inquiring, about phenomena involvesdescribing, defining, hypothesizing (i.e.,argumentation) and deriving generalizations(i .e., analysis ; Hillocks ,1982). Such strategiesenable understanding phenomena leading towriting improvements (Hillocks, 1986). Wedesigned inquiry-based writing activitiesand tested whether they improved writing(Hypothesis 1).

We also examined if the way studentsengaged in inquiry-based writing activitiescould have an impact. Students answered

Phanikiran Radhakrishnan, Department ofManagement, University of Toronto at Scarbor-ough. Ulrich Schimmack,Department of Psychol-ogy, University of Toronto Mississauga. DianneLam, Department of Psychology, University ofToronto at Scarborough.

Correspondence concerning this articleshould be addressed to Phanikiran Radhakrishnanat phanira@utsc.utoronto.ca

questions that help them engage in eachinquiry-based strategy or followed instruc-tions . For example, a question used to engageargumentation was "What results supportHypothesis X?" whereas an instruction was"Present at least one piece of evidence forHypothesis X". Further, because these ways(or designs) vary in how much they clarifythe demands of each inquiry-based strategy,they should vary in their impact on writingquality. Questions should be more effectivethan instructions because questions focusstudents' attention on the relevant featuresof each strategy, increasing students' un-derstanding of what is required from themin each strategy (Hypothesis 2). Researchshows that multiple-choice items framedas questions were less difficult than thoseframed as non-questions, which were harderto understand (Dudycha & Carpenter, 1973),more confusing and ambiguous (Violato &Marini, 1989). Similarly, Lawson, Bodle,Houlette, and Haubner (2006) show thatstudents leam best when they write answersto questions while viewing educational videosbecause questions mandate students to thinkdeeply about them.

We also hypothesized that practiceshould improve writing (Hypothesis 3). Prac-tice improves writing (Johnstone,Ashbaugh,& Warfield, 2002) because it helps acquireskills needed to do the task and proceduresrequired to execute them (Ericsson, Krampe,

247

248/ Journal of Instructional Psychology, Vol. 38, No. 4

& Tesch-Romer, 1993). Thus, we combinedthe practice conditions with the differentdesigns of writing activities. Students whoanswered questions while engaging in eachtype of inquiry got repeated opportunities topractice whereas those who received instmc-tions got only one opportunity to practiceeach type of inquiry and those in the controlgroup did not receive any.

We also explored the efficacy of provid-ing students with both instmctions and ques-tions . This group, compared to the group thatreceived only questions enabled us to test,in part, the separate effects of instructionsand questions. We predicted that repeatedopportunities for practice will only improvewriting if we provided students with an ef-fective strategy on how to do the writing taskwell. Giving students questions or providingthem with both questions and instructionswould teach them the best method to dothe task, leading to similar improvementsin wdting when compared to the group thatonly received instmctions (Hypothesis 4).

MethodParticipants

Participants (N=94) were undergraduatestudents in one of four, third-year psychologycourses. Two instmctors teaching two courseseach participated in our study. One instructorrandomly chose one course to be the controlgroup. The other three courses were exped-mental groups. We did not collect gender orage information because asking for it wasnot part of students' regular course activitiesand were not pertinent to our hypotheses.We asked for students' consent to use theircoursework for research'.

MeasuresTwo independent, trained raters assessed

the quality of summaries wdtten before andafterthe intervention .All participants summa-rized journal articles pertinent to the course^.Each summary was assessed on content (e .g.,

gives accurate, concrete and relevant detailsabout the results that support the hypothesis),grammar (e.g., consistency of tense withina sentence) and stmcture (e.g., no transitionstatement between paragraphs).

For rater training the first author andthe two raters graded 30 randomly selectedsummades and discussed inconsistencies. Allthree then graded 30 additional summadesafter which they reached an inter-rater reli-ability of .70. The two raters then marked theremaining summades^

ProcedureBased on Hillocks (1982), we designed

activities that would engage students in in-quiry (i.e., descdption, definition, argumen-tation and analysis)"*. The intervention wasconducted dudng a semester. All participantscompleted summades before and after theintervention as part of their course grade^.Participants in the first experimental groupwere given "Instmctions" to complete fourwdting assignments that allowed them topractice once, each of the four inquiry-basedstrategies (Single Tdal Practice with Instmc-tions Group). Participants in the secondexpedmental group were given "Questions"to complete nine wdting assignments thatallowed them to practice repeatedly, all ofthe four inquiry-based strategies (RepeatedPractice with Questions Group). Participantsin the third expedmental group (HybridGroup) received the manipulations of boththe first and second expedmental groups.They were given (a) Instructions to completefour wdting assignments that allowed themto practice once, each of the four inquiry-based strategies and (b) Questions to com-plete five wdting assignments that allowedthem to practice repeatedly all of the fourinquiry-based strategies. Participants in theRepeated Practice with Questions Group hadthe same number of assignments as those inthe Hybdd Group.

Inquiry-Based Thinking.. / 249

ResultsThe inter-rater reliability of the two inde-

pendent raters was .77 when collapsed acrosscontent, grammar, and stmcture. Therefore,we used an overall measure of writing qual-ity collapsed across the three dimensions inour analyses.

To test if those who received any type ofinquiry-based writing intervention (i .e., ques-tions or instmctions) wrote better summariesthan those who did not (i.e.. Hypothesis 1),we combined all experimental groups andcompared them with the control group. Weconducted a planned contrast between thepre and post-intervention summaries forthese two groups. Supporting Hypothesis1, the post-intervention summaries weresignificantly better for the combined ex-perimental groups (Mp̂ =̂ 6.86,Mp^ |̂= 7.27;i(74) = 3.456,p < .002') when compared tothe control group (M _.̂ = 6.46, M ^^- 6.67),which was not significant. Thus, students whodid not get any type of writing interventiondid not improve.

To test Hypothesis 2, we combinedthe two groups who received questions andcompared them with the group that receivedinstmctions. We conducted a planned contrastbetween the pre and post-intervention for thetwo groups. As predicted by Hypothesis 2,the post-intervention summaries were signifi-cantly better for the combined experimentalgroups that received questions (M ^̂ = 6.57,Mp̂ ,̂= 7.25; i(45) = 4.420,p < .OÔI) whencompared to the group receiving instmctions(Mp̂ =̂ 6.97, M^^^= 7.06) which was notsignificant. Thus, those who received ques-tions wrote better summaries than those whoreceived instmctions.

Hypothesis 3 states that those who hadrepeated opportunities to practice wouldwrite better summaries than those who hadone opportunity to practice who would, intum, write better summaries than those whohad none. Hypothesis 4 predicted that givenrepeated opportunities to practice, studentswho received only questions would write

Figure 1. Group differences in quality of writing.

Quality of

s.s-

BPreDPost

Control Instructions OnlySingle Trial

Questions Only InstructionsRepeated

Questions

Condition

250/ Journal of Instwctional Psychology, Vol. 38, No. 4

summaries of similar quality as those whoreceived both instructions and questions andthat both of these groups would be betterthan those who only received instructions.To test these hypotheses, we performed a4 x 2 mixed-design Analysis of Variance*.

We found a significant interactionbetween the type of experimental groupand the type of measure (i.e., pre vs. post-intervention): F{3,90) = 3.03,p < .04, rj^ =.06. Post-hoc t-tests demonstrated that thiseffect was for the Repeated Practice withQuestions Group if 22) = 2.62,;? < .02 and theHybrid Group i(22) = 3.58,/? < .003. Thus,partially supporting Hypothesis 3, studentswho had repeated opportunities to practicewrote better summaries after the interven-tion than those who had one opportunity topractice. However, those who had only oneopportunity to practice did not write bettersummaries than those who had none. Moreimportant, as predicted by Hypothesis 4,writing significantly improved (see Figure1) in both the conditions involving repeatedpractice with questions (i.e., the RepeatedPractice with Questions Group and the Hy-brid Group) but not in the condition involv-ing practice with instructions (Single TrialPractice with Instructions Group).

DiscussionWe investigated types of writing ac-

tivities that would most improve writing.We designed activities teaching students howto inquire based on previous research (Gra-ham & Perin, 2007; Hillocks, 1986) whichsuggests inquiry-based activities improveswriting because they increase understand-ing. We found, supporting our hypothesis,that inquiry-based writing interventionsimproved writing.

We further investigated how differentdesigns of inquiry-based writing activitieswould improve writing. Supporting our hy-pothesis and previous research (Dudycha &Carpenter, 1973; Violato & Marini, 1989),we found that giving students questions to

engage in the inquiry-based strategies is moreeffective than giving instructions. Questionsdraw students' attention to the demands ofeach strategy, increasing their comprehensionof how to engage in each strategy.

Next we examined whether the amount ofpractice could affect writing. We partially sup-ported our hypothesis: groups with repeatedopportunities to practice improved the most(see Johnstone et al., 2002 for similar find-ings). Practice enhances performance sinceit enables the acquisition of skills to do thetask and the procedures needed to executethem (Ericsson et al., 1993). However thegroup with only one opportunity to practicedid not do better than the control group thathad none. The former group may have hadinsufficient practice impeding the acquisitionof relevant skills and procedures.

Finally, we studied the joint effects oftheamount of practice and the different designsof writing activities. We demonstrated thatgroups with repeated practice in answeringquestions or in answering questions and fol-lowing instructions improved their writingwhereas the group with only instructionsdid not improve. Repeated practice improvesperformance only if leamers know how to per-form a task accurately (Ericsson et al., 1993).Thus, repeatedly answering questions trainsstudents on the best way to perform the taskwhereas only following instructions does not.

However the group that received onlyinstructions also had only one opportunity topractice. Because of the naturalized settingof our study, we could not compare studentswho got repeated opportunities to practicewith instructions to those who did so withquestions. Nevertheless, controlled laboratoryexperiments indicate that extended practiceleads to enhanced performance only if leamersare given suitable guidelines on how to do thetask (see Ericsson et al., 1993 for a review).Such findings further support our reasons forwhy the groups with repeated opportunities topractice with questions (or with questions andinstructions) improved the most. Therefore, we

Inquiry-Based Thinking.. / 251

validated our most important prediction thatthose who repeatedly practiced with questionswould do as well as those who had instruc-tions and questions but better than those whoreceived only instructions.

Although our quasi-experimental studyhad limitations such as non-randomized se-lection of participating instructors, courses,lack of control for course difficulty and par-ticipants, it had several strengths. A criticalaspect of our study is that we assessed writingbefore and after the intervention. Showingthat students wrote better after than beforethe intervention, establishes validity for ourfindings. Second, we conducted our studyin a real-world setting using a motivatedsample of leamers. Our students completedthe activities for their course grade and hence,had an incentive to do them well. They werealso from the same department and institu-tion, varying only in whether or not theyparticipated in courses with the intervention.Thus, we addressed a number of shortcom-ings identified by Johnstone et al. (2002) instudies on writing improvement'. Finally, byexamining the role of practice we contributeto an under-investigated aspect of researchon writing skill development (Kellogg &Raulerson, 2007). To conclude, our studyclarifies the features of writing activities thatwould improve writing: engage students ininquiry, require repeated practice, and involvethe answering of questions.

ReferencesDudycha, A. L., & Carpenter, J. B. (1973).

Effects of item format on item discrimination anddifficulty [Electronic version]. Journal of AppliedPsychology, 58,116-121.

Ericsson, K. A., Krampe, R. T, & Tesch-Romer,C.( 1993).Theroleof deliberate practice inthe acquisition of expert performance [Electronicversion]. Psychological Review, 100, 363-406.

Graham, S., & Perin, D. (2007). Writingnext: Effective strategies to improve writing ofadolescents in middle and high schools – A reportto Carnegie Corporation of New Kor̂ . Washington,DC: Alliance for Excellent Education.

Hillocks,G., Jr. (1982). Inquiry and the com-posing process: Theory and research [Electronicversion]. Coilege English, 44,659-673.

Hillocks, G.,h. (1986). Research on writtencomposition : New directions for teach / «g. Ill i nois :National Conference on Research in English.

Johnstone,K.M.,Ashbaugh,H.,&Warfield,T. D. (2002). Effects of repeated practice andcontextual-writing experiences on college stu-dents' writing skills [Electronic version]. Journalof Educational Psychology, 94, 305-315.

Kellogg,R.T.,&Raulerson,B.A.,III.(2007).Improving the writing skills of college students[Electronic version]. Psychonomic Bulletin &Review, 14, 237-242.

Lawson, T. J., Bodle, J. H., Houlette, M.A., & Haubner, R. R. (2006). Guiding questionsenhance student leaming from educational videos[Electronic version]. Teaching of Psychology,55,31-33.

The National Commission on Writing: InAmerica's Schools and Colleges. (2003, April).The neglected "R": The need for a writing revo-lution. Retrieved October 13, 2008, from http://www.writingcommission.org/prod_downloads/writingcom/neglectedr.pdf

Violato, C , & Marini, A. E. (1989). Effectsof stem orientation and completeness of multiple-choice items on item difficulty and discrimination[Electronic version]. Educational and Psychologi-cal Measurement, 49,287-295.

Footnotes'AH students' summaries were photocopied

before they were graded by the course TeachingAssistant. We used these copies for our analyses.

-Directions for the written summaries were:Your assignment is to summarize the article. Yoursummary should be 400 to 500 words long. Yoursummary should include enough detail so that thereader understands the theoretical background,major research questions or hypotheses, methods,results, and implications. This summary is muchlike an abstract, but is slightly longer.

'During practice and actual rounds of grad-ing, the first author and raters were blind to theconditions of each summary (pre/post, experi-mental/control) and the summaries were assessedrandomly.The only identifying information on thesummaries was the student numbers.

"•Descriptive writing demands narrating onlyconcrete and relevant details of the empirical

252/Journal of instructional Psychology, Voi. 38, No. 4

findings on a hypothesis. Definition entails iden-tifying essential features of a variable, providingexamples of it, and comparing and contrasting itwith a variable that it is typically confused with it(e.g., compare extraversion with agreeableness).Argumentative writing requires an explanation forwhy there is evidence for and against a hypothesis.Analytical writing entails the use of evidence tomake informed generalizations and derive em-pirically testable hypotheses (cf. Hillocks, 1982).

'Participants in all the experimental groupscompleted all the writing assignments as part oftheir course grade and received feedback after each.

Participants in the control group only receivedfeedback after the pre-intervention summary anddid not complete any of the intervening writingactivities that the experimental groups did.

'The pre and post-intervention measures werethe 2-level within-subjects factor. The between-subjects factors were the three experimental groupsand the one control group.

^They involve samples from different institu-tions that vary in the type of writing interventionsthey have, the student population, and the amountand style of feedback.

not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written

permission. However, users may print, download, or email articles for individual use.

not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written

permission. However, users may print, download, or email articles for individual use.