NIFL-ASSESSMENT 2005: [NIFL-ASSESSMENT:920] RE: Voice in writing

Archived Content Disclaimer

This page contains archived content from a LINCS email discussion list that closed in 2012. This content is not updated as part of LINCS’ ongoing website maintenance, and hyperlinks may be broken.

From: Dianne Glass (dglass@ksbor.org)
Date: Thu Feb 17 2005 - 14:55:46 EST


Return-Path: <nifl-assessment@literacy.nifl.gov>
Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id j1HJtkC28926; Thu, 17 Feb 2005 14:55:46 -0500 (EST)
Date: Thu, 17 Feb 2005 14:55:46 -0500 (EST)
Message-Id: <s214a2f1.000@ksbor.org>
Errors-To: listowner@literacy.nifl.gov
Reply-To: nifl-assessment@literacy.nifl.gov
Originator: nifl-assessment@literacy.nifl.gov
Sender: nifl-assessment@literacy.nifl.gov
Precedence: bulk
From: "Dianne Glass" <dglass@ksbor.org>
To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov>
Subject: [NIFL-ASSESSMENT:920] RE: Voice in writing
X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-Mailer: Novell GroupWise Internet Agent 6.5.1 
Status: O
Content-Length: 12519
Lines: 374

Kansas has used the CASAS Functional Writing Assessment (FWA) for almost
10 years.  While it requires an enormous commitment of time and energy
to ensure that the scoring of a performance-based assessment is
standardized, Kansas adult educators have responded positively to the
lengthy process of being "certified" to use the FWA and to maintaining
certification.  They report that the process has helped them become much
better teachers of writing.  


Dianne S. Glass
Director of Adult Education
Kansas Board of Regents
1000 SW  Jackson Street, Suite 520
Topeka, KS  66612-1368
785.296.7159
Phone:  785.296.7159
FAX:  785.296.0983
dglass@ksbor.org



>>> ltaylor@casas.org 2/17/2005 12:18:12 PM >>>
Marie,

Howard has articulated the main reason that the CASAS rubric is for
both ABE
and ESL learners. He said, "We don't hold learners to different
standards.
Our instructors see 'good writing' as 'good writing' whoever is doing
the
writing." 

We would add that employers and others on the receiving end of our
students'
writing don't have different standards, either. 

We would recommend placing ESL and ABE students in different classes
since
instruction and the kinds of strengths and errors will be very
different for
the two groups, but the general characteristics of writing for both
groups
can be described within a single rubric. We have been working with this
for
nearly ten years and have become very comfortable with scoring both
types of
learners on the same rubric, though it is often necessary to be careful
not
to over-reward ESL learners for "trying" when they haven't quite
succeeded
in writing at a certain level.

In answer to your earlier questions about writing prompts, I can
respond
with respect to the CASAS Functional Writing Assessment Picture Task,
which
is currently being used for accountability reporting in Kansas, Iowa,
Connecticut, Oregon, Indiana, Vermont and New York Even Start. Prompts
for
this task are line drawings showing a scene with a central critical
incident
as well as a number of other things happening in the picture. This type
of
prompt can be answered by students from beginning to advanced levels in
ABE,
ASE and ESL programs.

It takes a long time to develop a viable prompt, with many rounds of
revisions based on field-testing input from teachers and students and
back
and forth work with an artist. They are written by a small team of
test
developers who have extensive experience as adult ed. teachers. Topics
for
the prompts come from needs assessments from adult ed. programs and
workplace surveys. We currently have seven prompts - four that are on
general life skills topics (a car accident scene, a grocery store
check-out
scene, a park scene, and a department store scene). There are three
more
prompts that have a workplace focus - a restaurant kitchen scene, a
hotel
scene and a warehouse scene. 

Like the REEP, these prompts are scored with an analytic rubric, but
with
slightly different categories: Content; Organization; Word Choice;
Grammar
and Sentence Structure; and Spelling, Capitalization and Punctuation.
The
categories are weighted, with more importance given to the first three
categories to emphasize the importance of communication of ideas in
writing.
We have recently completed a study to convert the rubric scores to a
common
IRT scale, which provides a more accurate means of reporting results
across
prompts. We have also just completed a cut score study to refine the
relationship of the CASAS Picture Task writing scores to the NRS
levels. 

With all of the work that goes into developing and standardizing a
test
prompt, it is not made available for classroom practice. However, we
have
found several published materials that contain similar types of
pictures
that can be used for classroom practice. 

We encourage programs to share the rubric with students for
instruction, in
addition to using it to communicate test results to teachers and
learners.
Many teachers tell us that completing the training for the writing
assessment, which focuses on the scoring rubric, has given them a
better
understanding of how to approach the teaching of writing. The analytic
rubric provides clear diagnostic information about students' strengths
and
weaknesses in the different rubric categories.

I am very pleased that some states are choosing to include writing in
the
mix of assessments that can be reported for accountability purposes. It
is
more work to include performance assessment in a state's
accountability
system, due to the additional training and scoring demands, but the
states
that are doing it have found it to be worth the extra effort.

Linda Taylor, CASAS
(800) 255-1036, ext. 186


-----Original Message-----
From: nifl-assessment@nifl.gov [mailto:nifl-assessment@nifl.gov] On
Behalf
Of Marie Cora
Sent: Thursday, February 17, 2005 9:54 AM
To: Multiple recipients of list
Subject: [NIFL-ASSESSMENT:914] RE: Voice in writing

Hi Bonnie, thanks for this.

Yes, I think that it would have been real tricky for me to have a
rubric
that didn't distinguish between ESOL/ABE students.  Unless they are
transitioning from ESOL to ABE perhaps.  It's tricky enough, as you
note, to adhere to rubric anchors and so forth, so adding that you are
working with different populations with the assessment would add a
layer
that I would also find difficult.

CASAS folks:  can you tell us why the writing rubric is not separate?
What's the rationale there?  It seems like the needs, esp. at the
lower
levels, would be very different.

REEP folks:  what do you think about that?  Perhaps that was never a
consideration for you though, since REEP serves the ESOL population
(is
that right?).

Thanks,
marie

-----Original Message-----
From: nifl-assessment@nifl.gov [mailto:nifl-assessment@nifl.gov] On
Behalf Of bonniesophia@adelphia.net 
Sent: Tuesday, February 15, 2005 1:21 PM
To: Multiple recipients of list
Subject: [NIFL-ASSESSMENT:908] RE: Voice in writing

I, too, have been intrigued by the idea of "voice" in the rubric, and
while
I intuitively "know" what it means, I'm interested as an emerging
writing
specialist as to what elements would constitute voice, beyond more
traditional "academic" ways of "measuring" it. I think of the clarity
or
persuasiveness of a point of view supported with meaningful examples,
the
personal voice in a narrator struggling with complex questions,
forthright
emotion strikingly articulated with imagery or other means, an attempt
at
critical thinking, or "learning to learn," self-reflectiveness... I'd
be
interested in hearing from others.
Another point I encountered when I was involved with CT's working with
the
CASAS writing assessments: the rubric was not meant to distinguish
between
ABE and ESL students. As an evaluator, I as an ESL specialist was at a
disadvantage: having attained a certain level of skill in
"translating"
English learners' language into meaningful utterances, I'd
automatically
bring that to my evaluation: it was extremely difficult to adhere to
the
rubric controls and anchors, and not want to commend the ESL learner
for
attempting with limited language ability to voice something difficult
to
articulate in another language, as having communicated more than in
fact
they did. 
Best,
Bonnie Odiorne, Ph.D.
Writing Center, English Language Institute
Post University, Waterbury, CT

Original Message:
-----------------
From: Marie Cora marie.cora@hotspurpartners.com 
Date: Tue, 15 Feb 2005 12:02:28 -0500 (EST)
To: nifl-assessment@literacy.nifl.gov 
Subject: [NIFL-ASSESSMENT:906] RE: Voice in writing


Hi everyone,

A couple of observations:  

First, please do note that this assessment is a fine example of a
performance-based assessment that has been standardized.  So if anyone
still thinks that standardized assessments all look like TABE,
consider
your myth debunked.

I think that capturing voice in writing is quite important, and I'm
glad
that the REEP rubric includes this area.  If not for voice, the rest
of
the examination of the writing is based on the 'academics' of the
writing - and I feel like that leaves out the writer's (emerging)
personality.  I note in looking around a little bit, not a whole bunch
of other writing assessments take voice into account (the GED does not
for example). I also think that because voice is a dimension of the
rubric, students will pay more attention to that area and view it as
equally important as the other dimensions.  (A bit of "what counts
gets
counted" there.)

What do others think about voice and the other dimensions?

marie


-----Original Message-----
From: nifl-assessment@nifl.gov [mailto:nifl-assessment@nifl.gov] On
Behalf Of Suzanne Grant
Sent: Monday, February 14, 2005 1:38 PM
To: Multiple recipients of list
Subject: [NIFL-ASSESSMENT:905] RE: REEP Discussion Begins Today!

Greetings from Suzanne and Pat at REEP!

We would like to thank Susan O'Connor for her message,
congratulations,
and for getting this writing assessment discussion started. 

The development of our writing assessment, too, was also supported by
the Lila Wallace Foundation. From 1997-2002, the REEP Program was one
of
12 adult education programs nationwide funded to study what works in
assessment. The project was called the What Works Literacy Partnership
(WWLP) and was funded by the Lila Wallace Foundation. The lead agency
was Literacy Partners of New York. We had developed the rubric
earlier,
but through WWLP, we developed pre and post prompts and carried out
studies to determine the effectiveness of using the rubric to measure
progress.

Susan and all on this list, we would be interested in hearing what
writing traits you feel are important to include in writing assessment
rubrics. In our case, the engagement of the writer with the topic was
a
factor in how we were assessing the writing, and we, therefore, felt
we
needed to include voice as a trait in the REEP Writing Rubric.   

Other writing assessment questions and topics are welcome.

Suzanne Grant and Pat Thurston
REEP Writing Assessment Master Trainers
Arlington Education and Employment Program (REEP)
Arlington Public Schools
Arlington, Virginia

>>> S.Oconnor@BrooklynPublicLibrary.org 02/14/05 8:54 AM >>>
 Good Morning:  I wish to congratulate the REEP creators!  Brooklyn
Public
Library Literacy Program moved to writing in the early 90's.  In an
effort
to codify students' gains and with a grant from the then Lila Wallace
foundation we created a writing rubric for non-reading Adults - up to
about
a fifth grade reading level.  We have been using this successfully for
years.  But because it was not normed we couldn't use it to show gain
in
an
NRS environment.  

I have downloaded your article and handed it off to the folks at the
New
York State Education Dept.  I found this all very exciting.

Susan K. O'Connor
Brooklyn Public Library
Literacy Program Manager

-----Original Message-----
From: Marie Cora
To: Multiple recipients of list
Sent: 2/14/05 8:41 AM
Subject: [NIFL-ASSESSMENT:900] REEP Discussion Begins Today!

Good morning, afternoon, evening, and Happy Valentine's Day to you
all!



Please join us this week for our discussion of the REEP Writing
Process
and Rubric with Suzanne Grant and Pat Thurston, of the Arlington
Education and Employment Program (REEP) in VA.  The full information
and
the suggested preview resources for the discussion are listed for you
below.  Also, I wanted you to recall that Suzanne and Pat will be
accompanied by several trainers-in-training of the REEP process.
 
Welcome colleagues from REEP!!
 
February 14 - 18
Topic: Assessing Writing, Developing Rubrics, and Developing Effective
Writing Tasks
Guests:  Suzanne Grant and Pat Thurston, REEP Master Trainers
 
Recommended preparations for this discussion:
 
"The REEP Writing Story" at http://vawin.jmu.edu/vertex/article.php?v=1

<http://vawin.jmu.edu/vertex/article.php?v=1&i=1&a=2> &i=1&a=2
which discusses the development of their writing process and the
accompanying rubric.
 
"Making Sense of the REEP" at
http://www.sabes.org/resources/adventures/vol15/15teller.htm 
which discusses one program's experience with and reflections on using
the REEP process.
 
 
 
marie cora
Moderator, NIFL Assessment Discussion List, and 
Coordinator/Developer LINCS Assessment Special Collection at 
http://literacy.kent.edu/Midwest/assessment/ 
 
 
 
marie.cora@hotspurpartners.com 
 
 



--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .



This archive was generated by hypermail 2b30 : Mon Oct 31 2005 - 09:48:46 EST