[Assessment 1334] Re: Tests vs. Self Assessments of Literacy

Archived Content Disclaimer

This page contains archived content from a LINCS email discussion list that closed in 2012. This content is not updated as part of LINCS’ ongoing website maintenance, and hyperlinks may be broken.

tsticht at znet.com tsticht at znet.com
Mon Jun 9 14:12:42 EDT 2008


David: Regarding your questions about adult literacy (L) and numeracy (N)
needs, I have posted your questions and my answers below.

Question: Why do countries assess literacy (L) and numeracy (N) need?

Answer: To find out how many adults have poor LN abilities and who might,
therefore, benefit from education to improve their skills. This may lead to
policies providing for education provision to meet the need for LN
education.

Question: What have they learned?

Answer: Based on standardized tests of literacy (L) and numeracy (N) they
have learned that LN skills are distributed in the adult population from
low to high, particularly if the tests are designed to create such a
distribution (resembling to some extent the familiar bell curve). They have
also learned that most adults do not think they have major problems with
literacy and numeracy, though more, primarily immigrants, think they could
use better English language skills. They have also learned about a number of
correlations with test scores, e.g., unemployment, family background, etc.


Question: What is the added value of these assessments to low-literacy or
low-numeracy residents?

Answer: This is hard to say. In England the assessments added more to those
with medium to higher LN skills as the government moved to increase the
numbers of adults with educational qualifications, similar to passing one
or more of the GED tests in the U.S. and Canada. In the U.S. following the
NALS of 1993 and the implementation of the National Reporting System in
1998, then numbers of enrollees in the Adult Education and Literacy System,
i.e., the programs funded in part through the Workforce Investment Act,
Title 2, fell from over 4 million per year to around 2.6 million in 2005,
the last date for which I have seen data. In England latest reports are
that adult education in further education courses have dropped by around
1.5 million in the last few years, though this does not exactly focus on
those with low LN.

Questions: What is the added value of these assessments to their governments
and societies? Have the data had an impact on policy (yet)? What has been
the "return on investment" for national/international needs assessments of
adult literacy and numeracy?

Answer: Again, these are difficult to answer questions. The added value to
governments and societies has not been reported as far as I know. The data
have had an impact on policies for adult LLN provision in Australia and
Canada to less degree than in England and New Zealand and has had very
little impact on adult LLN policies at the federal level in the U.S., if by
policy one refers to added investment in the AELS. The NALS data has had a
bit more impact on funding policies in various states I believe. No "return
on investment" for assessments has been calculated for the nations I have
studied.

Question: Why are we measuring need but not demand?

Answer: In my 1998 paper on Beyond 2000: future directions for adult
education, I provided data showing that "demand" or "need" as indicated by
enrollments, had increased by about 100,000 a year from 1966 to reach a
level of some 4 million in 1998. I thought this illustrated the value of
the AELS and expected that with increased resources the growth would
continue up to 5 million or more. This would provide a strong basis for
arguing for increased funding for the AELS. But the imposition of the NRS
accountability requirements, with heavy demands on testing and data
collection, led to a precipitous drop in enrollments of over 1.5 million
which persists up to the present. I am all in favor of making a
demand-based argument for AELS support and enrollment data have served that
purpose in the past.

Questions: What percent or number of people see that they have basic skills
needs -- and want to pursue classes, tutorials or online learning to
address
it? Would that be a more useful figure for public policy decisions
than needs data?

Answer: The percent of people who see that they have basic skills needs and
want to pursue LN education IS a type of "needs" data. The question should
be: is this a better indicator of needs than the NAAL or similar national
standardized tests distributions of scores. I think it would be a more
useful figure for public policy decisions than NAAL-type assessments which
are subject to many, many arbitrary decisions in their construction of test
score distributions and more such decisions in deciding at which level or
levels the statements of "need" should fall. Many have used the present
NAAL levels of Basic and Below Basic to declare some 93 million adults
incapable of meeting their daily needs for work , living, and health
despite a complete lack of any basis for defining the levels using these
terms and for setting societal incompetence at these levels. Different
terms were used to refer to the earlier NALS levels, illustrating the
arbitrary nature of these level names, and the NALS used different
statistical data for making literacy levels than the NAAL, again
illustrating the arbitrary nature of the test development and
interpretation.

Questions: Some states have begun to count the number of people who want
classes but are put on waiting lists. This is one measure of demand.
Should these states be supported to improve/refine their data collecting
and reporting so we can have confidence in this measure, so it can be used
for decision making? Should we be looking at other measures of demand? If
so, what?
Should we do latent demand studies in adult education? ("If English classes
or GED Prep -- were offered on Saturday mornings would you attend?") If so,
should we do these studies systematically? Perhaps we should work on
addressing the actual demand staring us in the face and worry about
measuring latent demand -- and need -- later.

Answer: National, state and local needs and wants surveys asking people
about their skills needs and wants, using direct questioning and "latent
needs" questioning, seem like a good idea to me. There should be some
systematic studies along these lines and some repeated local needs/wants
assessments from time to time to reach the local folks would be useful. I
think that some adult educators should work on addressing the actual local
demand for provision right now, while others, perhaps those mostly committed
to research, might work on measuring directly stated and latent needs for
policy making at state and national levels. I have demonstrated how this
might be accomplished using telephone surveys in the U.S. and such surveys
have also been used in England and Canada.

Tom Sticht