NIFL-ASSESSMENT 2004: [NIFL-ASSESSMENT:485] RE: 400+

Archived Content Disclaimer

This page contains archived content from a LINCS email discussion list that closed in 2012. This content is not updated as part of LINCS’ ongoing website maintenance, and hyperlinks may be broken.

From: Jeri Levesque (levesqjr@webster.edu)
Date: Thu Apr 08 2004 - 13:18:41 EDT


Return-Path: <nifl-assessment@literacy.nifl.gov>
Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id i38HIfm23583; Thu, 8 Apr 2004 13:18:41 -0400 (EDT)
Date: Thu, 8 Apr 2004 13:18:41 -0400 (EDT)
Message-Id: <4075860C.4010807@webster.edu>
Errors-To: listowner@literacy.nifl.gov
Reply-To: nifl-assessment@literacy.nifl.gov
Originator: nifl-assessment@literacy.nifl.gov
Sender: nifl-assessment@literacy.nifl.gov
Precedence: bulk
From: Jeri Levesque <levesqjr@webster.edu>
To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov>
Subject: [NIFL-ASSESSMENT:485] RE: 400+
X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Status: O
Content-Length: 15954
Lines: 398

Laura,

Just a thought about subjective work and objective evaluation. 

Last year I attended a meeting hosted by the USED.  Most of the projects 
in this national program reported outputs.  The mandate to shift to 
outcomes was met with significant resistance because much of the work in 
many projects involved disseminating information to change parents' 
attitudes. During the meeting, a senior program officer rebuked all but 
17 of 88 studies the main speaker had collected to support the benefits 
of parental involvement in children's education.  At first I was 
stunned. It seemed we were being ever more pressured to connect outcomes 
with scientific research while working with less and less resources for 
high level evaluation and assessment.  Then the light bulbs flashed. If 
the 17 studies carried the magic formula the administration was seeking, 
then by association, we evaluators just had to connect the dots between 
our attitude outcomes and research that said, "if so, then these things 
are related." I happily began connecting the "dots" between outputs, the 
few outcomes we'd captured, and the goals sought by the funding source 
through the "gold standard" research.  I used this logic in our next 
funding proposal. Our subsequent three year project was funded. No they 
didn't award us gold, but we earned enough copper to stay afloat.

If your work is subjective, so be it. There is always a connection 
between attitude and knowledge, attitude and skills, attitude and 
behavior.  Just connect the dots with research that says these things 
make changes for the better and meet the project's goals.

Best,

Jeri Levesque, Ed.D.
Associate Professor, Director, Webster Institute for Literacy
Webster University
St. Louis, MO 63301

Innis, Laura (DPH) wrote:

>Good morning~This is very helpful.  I am in touch with Sabrina Kurtz from
>World Ed (HEAL)(someone else had mentioned)  but I have not communicated
>with Marsha.  I forwarded Jeri's email to our evaluator who is also excited
>about all these websites.  You are so right about outcomes.  All of our
>outreach programs were defunded after we started collecting/improving
>data/evaluation/reporting ~they did not appear to be successful.  It is also
>very difficult for us as health educators and outreach workers to "hear"
>that we need to change the way we've always done things and to incorporate
>accountability when so much of our work seems subjective.  We see first hand
>how much it means to people but as a funder I realize how scarce funds are,
>I know that assessment is important.  Again, thanx everyone for the
>feedback.  Laura
>
>-----Original Message-----
>From: Marie Cora [mailto:mariecora@hotmail.com]
>Sent: Thursday, April 08, 2004 10:46 AM
>To: Multiple recipients of list
>Subject: [NIFL-ASSESSMENT:482] RE: 400+
>
>
>Hi Jeri, thanks for your reply.
>
>Good call on Marcia's NIFL fellowship - I had forgotten - Marcia is the 
>person who developed the Student Health Education Teams that I mentioned 
>yesterday.
>
>Thanks SO MUCH for this great listing of further resources - Laura, let us 
>know if you find some of what you're looking for.
>marie cora
>NIFL Assessment List Moderator
>
>
>  
>
>>From: Jeri Levesque <levesqjr@webster.edu>
>>Reply-To: nifl-assessment@nifl.gov
>>To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov>
>>Subject: [NIFL-ASSESSMENT:479] 400+
>>Date: Wed, 7 Apr 2004 18:11:50 -0400 (EDT)
>>
>> Laura,
>>
>>I am an evaluator evaluation coordinator for Missouri's state literacy 
>>resource center and local Even Start family literacy programs. I also teach
>>    
>>
>
>  
>
>>evaluation at the graduate level. I'm answering Jane's call to respond to 
>>your posting.
>>
>>The current GPRA direction is aimed at performance outcomes and program 
>>impacts.  The model currently in vogue is the logic model.  Your concern 
>>about putting things in writing is valid, because federal programs that 
>>can't demonstrate measured outcomes, as opposed to program activity 
>>(outcomes) are in jeopardy. One example is the Even Start program that is 
>>in danger of being zero budgeted because while it consistently produced 
>>outputs it did not produced enough outcomes to suit the administration.  
>>Your topic;
>>
>>I am currently trying
>>to evaluate the success of a peer education curriculum designed to motivate
>>women to get breast and cervical screenings
>>
>>is one I am fairly familiar with, I did a series of presentations for our 
>>project here in Missouri.  I believe Marsha Drew Hohn also did some amazing
>>    
>>
>
>  
>
>>work in Lowell, Mass in a family literacy program, on the same subject. She
>>    
>>
>
>  
>
>>published her work as a NIFL Fellowship Report, I think you can order it 
>>    
>>
>>from ed.gov/pubs to see what measures she used.
>  
>
>>The outcomes you want to consider are women's knowlege, attitudes, skills, 
>>and behaviors associated with BCS.  Is there a correlation between knowing 
>>how important it is to do screening and then actually do monthly BSE's and 
>>get regular mams and paps?  That's the real connection you're trying to 
>>document.
>>
>>I suggest you start by reading Marsha's outstanding report (it is online at
>>    
>>
>
>  
>
>>NIFL but it's a pretty clunky set of PDFs) and then giving her a call.
>>
>>I've also included some web sites to help you to consider different 
>>measures and evaluation designs.
>>
>>
>>
>>  1. American Evaluation Association
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=1&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D57f5faff626e8334%26click
>edItemRank%3D3%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%25
>2Fwww.eval.org%252F%26invocationType%3D-%26fromPage%3DNSCPIndex2%26amp%3Bamp
>Test%3D1&remove_url=http%3A%2F%2Fwww.eval.org%2F>
>  
>
>>     Devoted to the application and exploration of evaluation in all
>>     its forms. ... Click here to go directly to the Evaluation 2004
>>     website. ...
>>     http://www.eval.org/
>>  2. Evaluation Criteria from "The Good, The Bad & The Ugly: or, Why
>>     ...
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=1&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D57f5faff626e8334%26click
>edItemRank%3D6%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%25
>2Flib.nmsu.edu%252Finstruction%252Fevalcrit.html%26invocationType%3D-%26from
>Page%3DNSCPIndex2%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Flib.nmsu.edu%2
>Finstruction%2Fevalcrit.html>
>  
>
>>     A easy to use guide for web evaluation. Lists evaluation criteria
>>     with links to actual pages that illustrate each point. ...
>>     Evaluation Criteria. ...
>>     http://lib.nmsu.edu/instruction/evalcrit.html
>>  3. evalwebs
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=2&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D59bf6f74fc9838eb%26click
>edItemRank%3D13%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%2
>52Fgsociology.icaap.org%252Fmethods%252F%26invocationType%3Dnext%26fromPage%
>3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fgsociology.icaap.
>org%2Fmethods%2F>
>  
>
>>     ... Resources for Methods in Evaluation and Social Research. This
>>     page lists FREE resources for methods in evaluation and social
>>     research. ...
>>     http://gsociology.icaap.org/methods/
>>  4. Resource Selection and Information Evaluation
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=3&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c5510%26clicked
>ItemRank%3D23%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252
>Falexia.lis.uiuc.edu%252F%257Ejanicke%252FEvaluate.html%26invocationType%3Dn
>ext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fal
>exia.lis.uiuc.edu%2F%7Ejanicke%2FEvaluate.html>
>  
>
>>     Resource Selection and Information Evaluation. ... Resource
>>     selection and information evaluation involves answering three main
>>     questions. ...
>>     http://alexia.lis.uiuc.edu/~janicke/Evaluate.html
>>  5. WWW Virtual Library: Evaluation
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=3&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c5510%26clicked
>ItemRank%3D27%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252
>Fwww.policy-evaluation.org%252F%26invocationType%3Dnext%26fromPage%3DNSCPNex
>tPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.policy-evaluation.org%
>2F>
>  
>
>>     The World Wide Web Virtual Library: Evaluation. The Social
>>     Policy-Evaluation area of the the World-Wide Web Virtual Library.
>>     Links ...
>>     http://www.policy-evaluation.org/
>>  6. The Evaluation Center
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=3&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c5510%26clicked
>ItemRank%3D28%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252
>Fwww.wmich.edu%252Fevalctr%252F%26invocationType%3Dnext%26fromPage%3DNSCPNex
>tPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.wmich.edu%2Fevalctr%2F
>  
>
>>     The Evaluation Center: Provides information about Center projects,
>>     publications, staff, history, and support services, including the
>>     Directory of Evaluators ...
>>     http://www.wmich.edu/evalctr/
>>  7. Practical Assessment, Research and Evaluation
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=3&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c5510%26clicked
>ItemRank%3D29%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252
>Fpareonline.net%252F%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp
>%3BampTest%3D1&remove_url=http%3A%2F%2Fpareonline.net%2F>
>  
>
>>     ... on-line journal provides education professionals access to
>>     refereed articles that can have a positive impact on assessment,
>>     research, evaluation, and teaching ...
>>     http://pareonline.net/
>>  8. Online Evaluation Resource Library
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=4&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D59bf6f74fc984211%26click
>edItemRank%3D37%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%2
>52Foerl.sri.com%252F%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp
>%3BampTest%3D1&remove_url=http%3A%2F%2Foerl.sri.com%2F>
>  
>
>>     home, : Online Evaluation Resource Library. ... Welcome to OERL,
>>     the Online Evaluation Resource Library. ... OERL has been serving
>>     the evaluation community for 4 years. ...
>>     http://oerl.sri.com/
>>  9. Basic Guide to Program Evaluation
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=5&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D59bf6f74fc9849b7%26click
>edItemRank%3D46%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%2
>52Fwww.mapnp.org%252Flibrary%252Fevaluatn%252Ffnl_eval.htm%26invocationType%
>3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2
>Fwww.mapnp.org%2Flibrary%2Fevaluatn%2Ffnl_eval.htm>
>  
>
>>     Basic Guide to Program Evaluation Written by Carter McNamara, MBA,
>>     PhD | Applies to nonprofits and for-profits unless noted
>>     First-Timers | Library Catalog ...
>>     http://www.mapnp.org/library/evaluatn/fnl_eval.htm
>> 10. Rubrics
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=6&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c6acd%26clicked
>ItemRank%3D52%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252
>Fwww.ncsu.edu%252Fmidlink%252Fho.html%26invocationType%3Dnext%26fromPage%3DN
>SCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.ncsu.edu%2Fmidli
>nk%2Fho.html>
>  
>
>>     ... Rubrics and Evaluation Resources. ** NOTE: Download ... you.
>>     The form also provides for self-evaluation, as well as
>>     teacher-evaluation. You ...
>>     http://www.ncsu.edu/midlink/ho.html
>> 11. evaluation cookbook contents
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=7&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D59bf6f74fc98555e%26click
>edItemRank%3D63%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%2
>52Fwww.icbl.hw.ac.uk%252Fltdi%252Fcookbook%252Fcontents.html%26invocationTyp
>e%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F
>%2Fwww.icbl.hw.ac.uk%2Fltdi%2Fcookbook%2Fcontents.html>
>  
>
>>     A practical guide to evaluation methods for lecturers, for
>>     browsing online. A publication from LTDI. to ... INITIATIVE,
>>     Evaluation Cookbook. ...
>>     http://www.icbl.hw.ac.uk/ltdi/cookbook/contents.html
>> 12. MEASURE Evaluation - Home Page
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=7&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D59bf6f74fc98555e%26click
>edItemRank%3D64%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%2
>52Fwww.cpc.unc.edu%252Fmeasure%252F%26invocationType%3Dnext%26fromPage%3DNSC
>PNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.cpc.unc.edu%2Fmeas
>ure%2F>
>  
>
>>     Monitoring and Evaluation to Assess and Use Results. ...
>>     http://www.cpc.unc.edu/measure/
>> 13. User-Friendly Handbook for Project Evaluation:
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=8&offset
>>    
>>
>=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D57f5faff626eb9b2%26click
>edItemRank%3D79%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%2
>52Fwww.ehr.nsf.gov%252Frec%252Fprograms%252Fevaluation%252Fhandbook%252F%26i
>nvocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_ur
>l=http%3A%2F%2Fwww.ehr.nsf.gov%2Frec%2Fprograms%2Fevaluation%2Fhandbook%2F>
>  
>
>>     User-Friendly Handbook for Project Evaluation: ... Table of
>>     Contents & Introduction to the User-Friendly Handbook for Project
>>     Evaluation: ...
>>     http://www.ehr.nsf.gov/rec/programs/evaluation/handbook/
>> 14. The Evaluation Exchange - at the Harvard Family Research Project
>>     ...
>>     
>><http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=10&offse
>>    
>>
>t=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c7f9d%26clicke
>dItemRank%3D92%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%25
>2Fwww.gse.harvard.edu%252Fhfrp%252Feval.html%26invocationType%3Dnext%26fromP
>age%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.gse.harva
>rd.edu%2Fhfrp%2Feval.html>
>  
>
>>     Harvard Family Research Project's free periodical, The Evaluation
>>     Exchange, provides a forum for knowledge-sharing among evaluators,
>>     policymakers, and ...
>>     http://www.gse.harvard.edu/hfrp/eval.html
>>
>>McREL and others have free publications (print) on evaluation - I have a 
>>bunch of other resources that I'd be happy to share.
>>
>>Jeri Levesque, Ed.D.
>>Associate Professor, Director, Webster Institute for Literacy
>>Webster University
>>St. Louis, MO
>>
>>
>>North Central Regional Education Lab   Evaluation Reports     
>>http://www.ncrel.org/re/mdade/
>>
>>
>>Schwerdtfeger, Jane wrote:
>>
>>    
>>
>>>Laura, your topic sounds really interesting.  It would be a great topic to
>>>explore, and maybe we could get some evaluators or psychometricians who
>>>focus on educational assessment to comment on the discussion too.
>>>
>>>Jane Schwerdtfeger
>>>
>>>
>>>
>>>
>>>      
>>>
>
>_________________________________________________________________
>Persistent heartburn? Check out Digestive Health & Wellness for information 
>and advice. http://gerd.msn.com/default.asp
>
>
>  
>



This archive was generated by hypermail 2b30 : Thu Dec 23 2004 - 09:46:14 EST