Assessment in Distance Learning, July 9 - 13, 2007

Discussion Announcement

Dear Colleagues,

I'm pleased to announce the following Guest Discussion, which will be held during the week of July 9: Topic: Assessment in Distance Learning.

Guest Participants

Jennifer Rafferty currently works as an independent education consultant. Formerly, she worked at the Adult Literacy Resource Institute at SABES Boston where she held the role of Project Manager for the Massachusetts ABE Distance Learning Project. She has worked in the field of ABE since 1998.

Shannon Young is the Program Manager for the Support Center at Project IDEAL / AdultEd Online and is a Senior Research Area Specialist for the Program on Teaching, Learning, & Technology at The University of Michigan's Institute for Social Research. She oversees the Center's work on curriculum and assessment issues and leads the design team that creates online tools used to collect student data and support teachers in member states. Shannon has conducted education research in k-12 and ABE settings and been a tutor and writing instructor since 1991.

Recommended preparations for this discussion

Please consider any of the following questions as catalysts for discussion. There is some redundancy with the questions below; however, the questions are articulated slightly differently and so might provide people with different entry points of thought. Also, perhaps subscribers will find it interesting to view Massachusetts as a sort of case study given that the issues faced there appear to mirror the issues faced elsewhere in Distance Learning.

Measures of Progress-Informal and Ongoing

In the discussion on pre-assessments for online learning (see Resources of Interest, below), participants talked about the ways in which they use various pre-assessments, orientation activities, and technology training to gauge readiness and prepare their students to become distance learners who persist beyond the "try-out" phase.

In distance learning, the initial orientation period may be one of the few times teachers and students are face to face. The types of informal, ongoing assessments teachers routinely use in classrooms (e.g., "reading" students to determine comprehension, redirecting or expanding discussions, group activities, quizzes, etc.) frequently aren't available in a distance setting. So, once we have students in the distance-learning door, how can we use progress measures and informal assessments to help keep them engaged? What do those measures entail? What does and doesn't work and why? What role might partner agencies and/or interested "others" play in the assessment process and in helping students stay connected and engaged?

As a distance teacher, how do you use the results of progress measures to inform your teaching?

Post-Testing and the NRS

In June, the NRS posted its official guidelines for counting distance learners in the NRS tables. One of the guidelines for counting distance learners states that students be post-tested in a proctored setting using a standardized assessment at intervals designated by the test developer or by state policy. Some states follow test developer guidelines. Others mandate students be post-tested after a certain number of days.

• How do you get your distance learners to take standardized post-tests?
• How do you prepare them in advance to understand the necessity of post-testing?
• What sorts of partnerships have you formed with other service providers (e.g., libraries, career link centers, etc.) to handle post-testing distance learners? What advice would you have for others interested in forming these partnerships?
• What do you do when a student achieves his/her goal before the pre-determined post-test interval? For example, many students come into distance programs with specific, short-term goals and study intently for brief periods of time (e.g., 50 hours of study over a two-week period). They meet their objectives and are ready to exit a program. If they have not been designated as a project-based learner, how do you handle accounting for these students? Do you post-test even if they haven't met the pre-established time-based criteria?

The following questions are based on discussions that have come up with distance learning programs in Massachusetts:

Questions related to standardized assessments, NRS, and state assessment policies:

1. What standardized testing instruments do you use with distance learners in your state?
2. Are your distance learning programs required to follow assessment policies/guidelines that were created for classroom programs, or has your state developed a separate set of distance learning assessment policies?
3. Do the standardized assessments align well with the dl curricula being used in your state?
4. How successful has your program been with pre and post assessment of distance learners?
5. What strategies have you implemented to encourage learners to return for a post-test and which strategies have been the most successful?
6. Has your state developed any remote testing guidelines that would make it easier for learners to access required assessments without having to return to the adult learning program?
7. Does your state use a standardized assessment that is online/web-based? If so, has having a web-based assessment made it easier for your program to pre and post-test learners?

Questions related to other forms of assessment for dl programs:

1. What other assessment instruments do you use to gauge whether your learners are appropriate for distance learning delivery?
2. Do you think that these assessments give a good indication as to whether or not the learner is prepared for self-directed learning?
3. What skills needed for dl are not so easily evaluated with inventories and assessments?

Resources of interest:

• Project IDEAL (Improving Distance Education for Adult Learners)
• AdultEdOnline (Introduction to distance teaching and free online self-assessment for classroom teachers interested in becoming a distance teacher)
• "Pre-Assessments for On-Line Learning" Excerpt from Special Topics Community Literacy discussion
  
  The discussion begins with questions regarding early drop-out and retention issues and asks whether "try-out periods" are integral to on-line learning processes. It then focuses on the inclusion of pre-assessments, as well as pre- and on-going orientation sessions as strategies for improving retention and success rates.

Please note: We do not control and cannot guarantee the relevance, timeliness, or accuracy of the materials provided by other agencies or organizations via links off-site, nor do we endorse other agencies or organizations, their views, products or services.