Skip to main content

ARCS Diagnostic Reading Assessment, 2005

ARCS Diagnostic Reading Assessment

The following Guest Discussion took place on the NIFL-Assessment Listserv between April 11 and April 15, 2005. The topic is the ARCS Diagnostic Reading Assessment.

Summary of Discussion

The Discussion focused on the importance and components of reading diagnostic tests; the use of ARCS as a reading assessment that can be used for instructional purposes; suggestions of various assessments for reading; and questions and comments by participants of the NCSALL study whose students' reading scores were used as ARCS data.

Initial questions posed to participants included:

  • What reading assessments do you use? Why?
  • How do you determine a student's reading level? (What reading diagnostics do you use?)
  • What are the challenges you face trying to assess reading?
  • What recommendations do you have for assessing reading?

A few participants who participated in the NCSALL (National Center for the Study of Adult Learning and Literacy) study that collected student samples for the development of the ARCS data set posed follow-up questions and commented on their experience working in the study. One of the questions asked was if ARCS could be used as a placement assessment. Rosalind's response noted that there are suggestions of diagnostic reading assessments in the Test Bank at the ARCS website (http://lincs.ed.gov/readingprofiles). In addition, she noted the importance of including several different measures in terms of a reading diagnostic including:

  • Selecting a diagnostic that focuses on reading sub-skills such as word recognition, word attack, oral reading accuracy, oral reading fluency, vocabulary, and silent reading comprehension;
  • listening vocabulary tests that reveal the level of word meanings a student brings to understanding text;
  • a questionnaire that provides information on the person's background and goals.

In addition, Rosalind noted that a minimum reading diagnostic would include a graded test of word recognition, a vocabulary assessment, and a silent reading comprehension test. Examples of various tests were provided in each of these areas.

The discussion continued in terms of the need for each student to have a reading profile for effective teaching to occur, and that some students will need more than initial diagnostics in order to pin-point difficulties in various reading sub-skills. Again, examples of assessments for use were provided. The discussion became focused on how assessments can be used for teaching purposes. Fluency was focused on in terms of the importance of oral versus silent reading, and it was noted that fluency must be measured in terms of oral reading, which is done relatively little in classrooms.


Guest Discussion Announcement

Hello fellow List members. I hope this email finds you well.

I'm pleased to announce the following Guest Discussion, which will begin on Monday of next week:

April 11 - 15

Topic: Using Diagnostic Reading Assessments: For whom, when to test,
and which to choose?

Guest: Rosalind Davidson, Research Associate/Lecturer on Education,
National Center for the Study of Adult Learning and Literacy (NCSALL),
Harvard University Graduate School of Education

Recommended preparations for this discussion:
Review the assessment website based on the Adult Reading Component Study at NCSALL. http://lincs.ed.gov/readingprofiles

I suggest that you click on Match a Profile, to learn about the 11 ARCS
-based profiles developed, and where your student might fall within the
profiles. Also, if you click on Take the Mini-Course, you will learn
about reading generally, assessing reading, and developing instruction.

Finally, here are some questions that I throw out to you all to consider in preparation for our discussion with Rosalind next week:

  • What reading assessments do you use? Why?
  • How do you determine a student's reading level? (What reading
    diagnostics do you use?)
  • What are the challenges you face trying to assess reading?
  • What recommendations do you have for assessing reading?

Thanks! I'm looking forward to seeing you all here next week!

marie cora

Moderator, NIFL Assessment Discussion List, and
Coordinator/Developer LINCS Assessment Special Collection at
http://literacy.kent.edu/Midwest/assessment/


Discussion Thread


Excellent!

I have been thinking about this -- again -- recently. Project RIRAL
(which participated in the NCSALL study) is re-organizing its
orientation to classes and placement assessments. My questions:

1 -- Should we use the ARCS recommendations to develop a placement
assessment into our ABE reading and Advanced ESL classes? If so, any
suggestions and / or helpful hints?

2 -- RIRAL's instructors are participating in a series of workshops to
understand standards-based education and to align curriculum and
instruction with that approach to learning. How does the ARCS see
itself supporting / fitting into standards-based instruction?

Howard Dooley, Project RIRAL


Hello Howard, Project Riral. Yes, your learners are part of the ARCS
Data set. We appreciate your very helpful cooperation in carrying out the ARCS work in Rhode Island.
About your issues -

  1. What do you use for intake placement now? TABE, Casas ...? anything else? What do they tell you that determines placement decisions? Why do you think you need more information?
  2. The most thorough assessment of English reading would be with a
    diagnostic test that gives information on many important reading
    sub-skills: word recognition, word attack, oral reading accuracy, oral
    reading fluency, vocabulary, and silent reading comprehension. The test we used for the ARCS is the Diagnostic Assessments of Reading (DAR).
    Another is the Quick Adult Reading Inventory, a shorter test of the same sub-skills (minus reading comprehension), but you have from the TABE or other mandated test you give. - See the Test Bank on the ARCS website.

    In addition, a listening vocabulary test such as the Peabody Picture Vocabulary Test would tell you the level of word meanings they bring to understanding text. For native English speakers, it is also a measure of cognitive power. For native Spanish speakers in either the ESOL or ABE program, there is the TVIP, a similar test that assesses Spanish vocabulary.

  3. At the least, you need a graded test of word recognition and a
    vocabulary assessment in addition to a silent reading comprehension test.
  4. Add a questionnaire that will tell you about past education, health,
    and the learner's reason for taking classes (goals). There is a
    downloadable questionnaire on the website.

As far as aligning curriculum with standards-based education - achieving fluent oral reading and silent comprehension is our main goal. In general, I don't have a problem with SBE as long as it can serve that goal. Is there a particular set of standards you are working with in your workshops?

Rosalind Davidson, NCSALL


Rosalind --

Thank you for your comments. I will share them with our instructors at
our meeting next week. I am sold on moving in this direction for our
ABE Reading and Intermediate ESOL classes. My reflection now is whether to require creating a profile for every learner, or whether to target learners who are having more difficulty learning in the given-class environment.

RIRAL currently uses the CASAS for placement and initial diagnostics.
Until this past year, individual instructors would follow up in-class
with a skills-based diagnostic. For this year, instructors met in
groups to determine guidelines for this follow-up diagnostic (as I call
it). For Reading, we use reading passages with a variety of questions,
multiple-choice and short answer. As you note, at best this assesses
silent comprehension.

Instructors are looking for information that will enable learners to
advance as quickly as possible to their goals -- which generally involve a next step, which we at RIRAL are, well, a bridge to: for example, to post-secondary ed, to a training program, to improved or more secure employment. So, we are targeting instruction, and building as solid and as broad a base in reading as we can, before the learners need (often psychologically) to move on.

The CASAS identifies priority competencies for learners to advance, but
not skills or standards. Yet. I am in contact with CASAS about their
on-going efforts to identify and support standards and skills. This
skills or standards information piece is something both instructors and
learners want. I think the ARCS provides a realistic, do-able structure for gathering this information in a way that is understandable for the instructor and the learner.

We have two sources for standards under consideration. RI recently had
a team from EFF come to talk about their Reading standard. Much of that work reflected and had the same base as the ARCS. It was easy to see how developing a reading profile, as you describe, could support the EFF Teaching/Learning cycle. I expect our instructors to have a similar workshop with a CASAS trainer this summer. I can see how broadening the reading instruction to include the elements in the ARCS would result in significant improvements on the CASAS.

Howard, Project RIRAL


Hi everyone,

Howard, I am intrigued about a comment you made:

"My reflection now is whether to require creating a profile for every learner, or whether to target learners who are having more difficulty learning in the given-class environment."

Can you and Ros tell us a bit more about that? Why would you not create a profile for each learner? Why do you feel you need to focus on either the whole or a target group? (why not both?) Is there a time factor involved?

Ros, can you tell us how you have defined the goal: achieve fluent oral reading and silent comprehension? Is that done via the various
assessments used, or is there a different way that that is defined
within ARCS?

Thanks!

marie cora


Hi,

Yes, I think everyone should have a reading sub-skills assessment at
intake. At the least, a profile made up of the five reading components
as on the ARCS website should be determined at intake for all. If you look at the webpage -

lincs.ed.gov/readingprofiles/FT_Compare_Profiles.htm

you will see that just having scores on a silent reading comprehension
test may put learners in the same classroom who have similar silent reading comprehension scores but very different instructional needs. Now, that may have to be the case anyway if literacy organizations cannot arrange for flexible classes, but in that case, teachers will see up front the individual needs of a diverse group for whom she will have to plan instruction. In each case, assessment would serve its true purpose: to aid instruction.

There will be learners who need more than just this initial assessment.

If they test very low on word recognition, a teacher would want to see what elements of word reading are causing difficulty by giving the learners a word analysis test such as the one on our website: Sylvia Greene's Informal Word Analysis Inventory --it's free and takes about 10-15 minutes to administer. Then the reader would practice those constructions that he has not mastered. There are other assessments or observations a teacher can call upon to pin point difficulties with other sub-skills.

Next issue: Do we set benchmarks for fluency and silent reading
comprehension? Oral reading rate is one part of fluency. A proficient adult reader can zip along reading accurately at 200 or more words a minute. Rate is tied to reading accuracy so that if someone's highest level of accurate reading is on a 6th grade equivalent passage, they should read at about 135 words per minute. The other part of fluency is reading smoothly with expression and intonation as if speaking. There are some measures to assess how smoothly a reader speaks a passage, but they are not necessary. Timing a reading (see the website on how to do it) and listening is sufficient. How high a level? Reading orally - accurately and smoothly - needs to be practiced at whatever level a learner is currently on. Often, too little oral reading goes on in classrooms. Fluent reading is descriptive of
good oral, not silent, reading.

Rate, as words per minute, and comprehension are measures of silent
reading proficiency. We head for high school, or GED, proficiency in silent reading comprehension.

Assessment and instruction follow each other in a continuous cycle until a goal is reached.

S'long all. Write all of you out there- How do you use assessments in
your teaching?

Rosalind Davidson


Hi Ros and everyone,

This is a very thorough assessment! I wonder if other folks who were
involved in the NCSALL study (as were Howard and RI) have questions or
can add comments about their experience with ARCS?

And to second Ros's question to you all: How do you use assessments in
your teaching? What does your reading program (curriculum and
assessment/s) look like?

Thanks!

marie cora

Moderator, NIFL Assessment Discussion List, and Coordinator/Developer

LINCS Assessment Special Collection at
http://literacy.kent.edu/Midwest/assessment/


Montana ABLE educators just had the opportunity two weeks ago to spend some time with Ros at the two Montana Route to Reading workshops. The
participants were extremely impressed with ARCS. At this time they are
beginning to experiment with many of Ros' recommendations for assessment and entering scores on the Match a Profile section.

Students in our programs are initially being assessed with the TABE and some type of word recognition test, i.e. WRAT, etc. In some programs if further diagnosis is needed, then students are given the DAR. Individual programs are then designed for each student. When possible, some small group instruction may take place; however, most of our instruction is individualized.

Thanks to Ros for coming out to Montana to share her expertise and to take a look at how a rural state operates! NP =)

Norene Peterson

Adult Education Center


Hello all,

Thanks Norene for hosting the ARCS presentations in Montana. I am
gratified to hear that participants in the workshops are using the website to diagnose their learners' reading.

Are other states where we have presented the ARCS putting the site to use? Is there anyone, or group, in Virginia, Texas, Tennessee, New York, New England states, Florida, Michigan, Oklahoma, Oregon, Nevada, using the ARCS, or a similar system to diagnose problems of poor readers? Let's hear from you -

Rosalind Davidson

Research Associate/Lecturer on Education

National Center for the Study of Adult Learning and Literacy

Harvard University Graduate School of Education

Nichols House - Appian Way

Cambridge, MA


How can a person or institute find out more about ARCS or even see a
sample of what the material looks like?

Katrina Hinson

The web site with ARCS information --

http://lincs.ed.gov/readingprofiles/MC_ARCS.htm

Cyndy Colletti

Literacy Program Manager

Illinois State Library Literacy Office

Gwendolyn Brooks Building

Springfield, IL


Hello Katrina -

To find out about the Adult Reading Components Study (ARCS), go to the ARCS website, http://lincs.ed.gov/readingprofiles. On the Home Page, click on either Match a Profile or on Take the Mini-Course. The next page will have a banner across the top listing several links on the site. Click on ARCS - and there you will be taken to information about the Study. The whole web site is based on findings from that study.

You will find descriptions of diagnostic materials used by ARCS as well as some tests you can download. Just go browsing - Let us know what you think - Do you find what you are looking for?

Rosalind Davidson


I've looked at the website I was referred to in an earlier email but I'm still not sure, other than the information contained for reading how to actually put that information to "use". I'm definitely interested in it because we don't do this at my school...the only assessment tool used is TABE for ABE/GED/AHS and CASAS for ESL students. As someone who encounters the reading difficulties in the classroom setting, it looks like the information on the website would be useful but I've yet to actually see an actual assessment tool beyond the WMT test that's there.

What am I missing?

Katrina Hinson


Katrina,

You need five Grade Equivalent scores: silent reading comprehension (you have that one from the TABE), word recognition (there is a link to the San Diego Quick Assessment on Make a Match page), graded Spelling lists are available from you local public schools (until we get one up on the site), oral reading rate (we tell you how to do it), and word meaning. Look on the site's Test Bank to find published tests on all the components; the prices are given also - Perhaps I am not answering your question - the site gives information about
learners who have similar patterns of scores on these sub-skills, but you have to do the assessment, and how to get the tests, not all the tests themselves, is given on the site. I suggest you begin at the beginning of the Match a Profile track. The Test Bank can be found from the Mini-Course track to Assessments to Test Bank.

Rosalind Davidson


Hello everyone,

I would like to thank Rosalind Davidson for being our guest last week.
Thanks to those of you who participated as well.

Let us know if any of you decide to try out ARCS, or if the folks who
are already using ARCS have any interesting info to report. We would
love to hear about it.

marie cora

Moderator, NIFL Assessment Discussion List, and
Coordinator/Developer LINCS Assessment Special Collection at
http://literacy.kent.edu/Midwest/assessment/