[Assessment 1333] Re: Tests vs. Self Assessments of Literacy

Archived Content Disclaimer

This page contains archived content from a LINCS email discussion list that closed in 2012. This content is not updated as part of LINCS’ ongoing website maintenance, and hyperlinks may be broken.

David J. Rosen djrosen at comcast.net
Mon Jun 9 09:26:30 EDT 2008

Tom, and others,

Thank you for pointing out the Henningsen research and asking for
additional research on adults' perceptions of their own literacy and
numeracy skills and the discrepancy between the NALS, NAAL, IALS, ALL
and adults' own perceptions. I would like to raise another set of
questions to consider about adult literacy and numeracy needs

Why do countries assess literacy and numeracy need? What have they
learned? What is the added value of these assessments to low-literacy
or low-numeracy residents? To their governments and societies? Have
the data had an impact on policy (yet)? What has been the "return on
investment" for national/international needs assessments of adult
literacy and numeracy?

Why are we measuring need but not demand? Even if 40-50% of a
population is in need, we can be certain that 40-50% will not be
interested in getting education services to address the need. What
percent or number of people see that they have basic skills needs --
and want to pursue classes, tutorials or online learning to address
it? Would that be a more useful figure for public policy decisions
than needs data? Some states have begun to count the number of
people who want classes but are put on waiting lists. This is one
measure of demand. Should these states be supported to improve/refine
their data collecting and reporting so we can have confidence in this
measure, so it can be used for decision making? Should we be looking
at other measures of demand? If so, what?

For example, in public transportation I understand that it is common
to do "latent demand" studies, that is, to survey people about
whether if a service were offered or extended they would take
advantage of it? ("If the #49 bus hour schedule were extended to
midnight instead of 11:00 PM would you use it at that time?") Should
we do latent demand studies in adult education? ("If English classes
-- or GED Prep -- were offered on Saturday mornings would you
attend?") If so, should we do these studies systematically?

Meanwhile, one thing we know for sure from the long waiting lists in
many states for English Language classes: there is a big demand for
these services, and the public sector has not come close to meeting
the actual demand, not to mention potential or latent demand for
these services. Perhaps we should work on addressing the actual
demand staring us in the face and worry about measuring latent demand
-- and need -- later.

David J. Rosen
djrosen at comcast.net

On Jun 8, 2008, at 5:45 PM, tsticht at znet.com wrote:

> June 8, 2008


> The Great Adult Literacy Skills Debate: Tests vs. Self Assessments


> Tom Sticht

> International Consultant in Adult Education


> In 2006, Inge Henningsen of the Department of Statistics in the

> University

> of Copenhagen presented a paper entitled: "Adults just don't know how

> stupid they are: Dubious statistics in studies of adult literacy and

> numeracy."

> (online at www.alm online.org/ALM13/programma%20alm13.pdf ).


> In this paper Henningsen comments on the many problems, conceptual,

> methodological, and statistical, with the International Adult Literacy

> Survey (IALS) of the mid-1990s and the Adult Literacy and

> Lifeskills (ALL)

> survey of 2003-06. One of the major factors in these assessments that

> Henningsen addresses is the finding in various nations of a wide gap

> between the literacy and numeracy

> skills of adults when the literacy test scores are taken as

> indicators of

> people's skill, and the skills that adults assign to themselves

> when asked

> to self-assess their literacy or numeracy skills.


> Though Henningsen focuses primarily on the data from these

> international

> adult literacy surveys for Denmark, similar gaps are found in

> various other

> nations. For instance, in Australia, based on the IALS test scores the

> report writers declared some 46 percent of adults to possess low

> literacy

> skills, whereas only 4 percent of the adults themselves thought

> they had

> low skills.


> In Canada and the United States, similar discrepancies were found,

> with 42

> percent of Canadians and 47 percent of U.S. adults being declared

> low in

> literacy based on the test scores, while only 5 percent of Canadian

> and 7

> percent of U.S. adults rated their literacy low.


> In New Zealand, using just the numeracy data from the ALL, 51

> percent of

> adults were declared low in numeracy based on test scores, while

> only 19

> percent rated their numeracy skills as low. Finally, in England,

> using a

> special test developed for the Skills for Life strategy in that

> nation, 16

> percent of adults were declared low in literacy based on their test

> scores

> but only 4 percent thought they had low test scores.


> Henningsen noted that the discrepancy between the self assessed

> proficiency

> and the conclusions based on test scores is not treated seriously

> in the

> reports and asks, "Is it ethically defensible to disregard the

> opinions and

> statements of the adults regarding their own skills and "narrate"

> big groups

> of adults in the labour market as excluded from society and lacking

> in basic

> skills." Answering this rhetorical question, Henningsen goes on to

> say, "I

> find it disturbing that the reports send the message that the

> experiences

> and assessments of the test persons themselves have no validity

> compared to

> the test results. Is it a viable for the adult education community

> to let

> surveys convey the impression that "adults just don't know how

> stupid they

> are."?


> One important consequence of adults' thinking that their literacy and

> numeracy skills are pretty good is that they will choose to not

> participate

> in language, literacy, and numeracy (LLN) provision to improve

> their skills.

> For instance, from various sources I can make rough estimates of the

> percentage of adults that the government says are in need of LLN

> provision

> that actually take part in LLN provision in a given year. In

> Australia the

> percentage of those the governments say are in need of LLN

> provision who

> actually enroll in LLN provision in a given year is around 4%, in

> Canada

> 10%, England 5%, New Zealand 11%, and the U.S. 3%. These (admittedly

> roughly estimated) percentages of participation are more in line

> with the

> self assessed needs of adults than the needs based on the paper and

> pencil

> tests.


> In the United States, the National Center for Education Statistics

> reports

> in the 2008 Conditions of Education that the percentages of adults

> aged 16

> or older who participated in adult education activities consisting

> of basic

> skills, English as a second language, or apprenticeships in 1995,

> 1999, 2001

> and 2005 were 3, 4, 4, and 3 percent respectively. Without the

> category of

> "apprenticeships" the percentages would be even lower. These low

> percentages of self reported participation in LLN are again more in

> line

> with the self assessments of adults regarding their literacy and

> numeracy

> skills than with the percentages declared to be "at risk" for low

> literacy

> based on the adult literacy survey tests.


> The large discrepancies between the percentages of adults needing

> basic

> skills education as given by governments based on the international

> adult

> literacy surveys, and the much smaller percentages of adults who

> perceive

> their literacy and numeracy skills to be so low that they are

> unable to

> progress in the societies in which they live pose problems for adult

> education. Some have suggested that adults may be too embarrassed

> to admit

> that they have a literacy or numeracy problem and that is why there

> is a

> large discrepancy between the adults' test scores and their self

> assessments of literacy. If this is so, then research is needed to

> establish that this is so. In general, major efforts are needed to

> better

> understand the genuine needs of adults for LLN provision, what

> sorts of

> educational programs would best meet these needs, and the sorts of

> activities that are needed to let

> adults understand the educational opportunities available to them.



> Thomas G. Sticht

> International Consultant in Adult Education

> 2062 Valley View Blvd.

> El Cajon, CA 92019-2059

> Tel/fax: (619)444-9595,

> Email tsticht at aznet.net





> -------------------------------

> National Institute for Literacy

> Assessment mailing list

> Assessment at nifl.gov

> To unsubscribe or change your subscription settings, please go to

> http://www.nifl.gov/mailman/listinfo/assessment

> Email delivered to djrosen at comcast.net

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lincs.ed.gov/pipermail/assessment/attachments/20080609/7945e0d3/attachment.html