English writing can be divided into different genres, such as description, exposition, narration, and argumentation. As one of the most widely-used writing genres, argumentative writing plays an important role in our daily life and academic study. People need to use argumentation whenever they need to resolve an uncertain thing, respond to an assertion or question, or challenge something which is believed to be certain by a significant party (Wu, 2008). Rorrenberg depicted this dialog nature of argumentation as “even when our audience is unknown, we write to persuade the unconvinced, to acquaint them with good reasons for changing their minds” (1997, p. 4). Ding et al. also claimed that the purpose of argumentation is to make the reader agree with its point of view and support it, to persuade him to change his mind or behavior, and to approve a policy or a course of action that it proposes (1994). Therefore, argumentation is a kind of oral or written dialogue with the audience, known or unknown, to convince them of a certain point of view.
2.2 Writing Ability Assessment
2.2.1 Classification of Writing Assessment Tasks
English writing assessment tests students’ English written expression ability according to specific test purposes or the ability standards stipulated in the syllabus and curriculum standards (Wu, 2002). Generally, there are two product-oriented approaches to writing tests: indirect writing assessments and direct writing assessments.
The indirect writing assessments adopt discrete test items to assess students’ knowledge of linguistic features, such as grammatical choices, error corrections, or even more specific writing behaviors, such as spelling or punctuation (Cumming, 1997). There are mainly three forms of indirect writing assessments: the multiple-choice type, controlled writing, and sequencing (Zhi, 2014). As the answer to indirect writing assessments is usually objective, indirect writing assessments can ensure the reliability of rating results and are time-saving. However, some scholars believed that writing ability was related to not only vocabulary and grammar, but also students’ personal traits, test circumstances, etc. Therefore, the indiscrete assessment was not communicative and could not reflect students’ writing ability comprehensively (Zhi, 2014).
Under such circumstances, more attention was paid to the direct writing assessments, in which students are required to produce a complete sample of writing. With this shift back to the direct writing assessments, the problems regarding content and construct validity were addressed (Knoch, 2009). To assess students' writing ability more comprehensively, this study selected the direct writing assessment. However, the direct writing assessments were also questioned in terms of the methods of collecting and rating the writing samples, such as the influence of rater subjectivity. In the next section, the scoring methods of direct writing assessments will be discussed in detail.
CHAPTER THREE RESEARCH METHODOLOGY ...................... 34
3.1 Research Questions .............................. 34
3.2 Research Participants ........................ 34
CHAPTER FOUR THE CONSTRUCTION OF THE COGNITIVE DIAGNOSTIC CHECKLIST AND Q-MATRIX .......... 44
4.1 The Construction of the Attribute Checklist ........................ 44
4.1.1 Analysis of Relevant Documents and Studies .................... 46
4.1.2 Analysis of Data from Expert Consultation I ....................... 47
CHAPTER FIVE DATA ANALYSIS AND DISCUSSION ................. 65
5.1 The Effectiveness of the CDA in Diagnosing Students’ Writing Ability ....... 65
5.1.1 The Validity of the Cognitive Diagnostic Checklist ........................... 65
