Discussion Announcement and Background
The Workforce Competitiveness Discussion List hosted a discussion on The Sectoral Employment Approach: The Promise and the Challenge from September 13 – 17, 2010 with guest facilitator Dr. Joshua L. Freely, Senior Research Associate & Director of Labor Market Research at Public/Private Ventures (P/PV). P/PV is a national nonprofit non-partisan organization that develops, tests and replicates innovative social programs to improve the lives of children, youth and families in high-poverty communities. Dr. Freely is an author of the study Tuning In to Local Labor Markets: Findings from the Sectoral Employment Impact Study (access the study at http://www.ppv.org/ppv/publications/assets/325_publication.pdf or the executive summary at http://www.ppv.org/ppv/publications/assets/327_publication.pdf.)
This research study examined three agencies with very different approaches to sector-focused training. The two year study followed 1,286 people who had participated in sector-focused training to study if the training made a significant difference in obtaining consistent employment, wages, or benefits for low-income, disadvantaged job seekers. Because the first year included the training, internships and job searches the most earning gains occurred in the second year. In the first year, the participants earned 18% more than members of the control group. The second year, this margin increased to 29%. This report compares the commonalities and challenges of the three agencies. The agencies used a mix of strategies and also a mix of both public and private funding sources. The results were positive for increased employment, wages, and benefits for the participants within the two years of the study.
Sector-focused training programs aim to connect disadvantaged job seekers and low-skilled workers to employment opportunities, addressing unmet hiring needs of local employers and improving participants'
prospects in the labor market. This study is the first random assignment evaluation of nonprofit-led sector-focused efforts, focusing on three distinct programs across the country: an employer/union association, a
social venture and a human service organization. These programs had strong effects for participants, including higher earnings and better jobs (as measured by hourly wages and access to benefits). As we emerge from the Great Recession, which has disproportionately affected disadvantaged workers, these strategies and the organizations that implement them may represent a key element in America's economic
recovery-for its workers and its employers.
As director of labor market research, Dr. Joshua L. Freely coordinates research activities for P/PV's labor market initiatives department. His areas of expertise are workforce development, the sociology of education, race and ethnic relations, evaluation and quantitative methods. Dr. Freely joined P/PV as a research associate in August 2006. Currently he is working on the Alternative Staffing Demonstration, Sectoral Employment Impact Study, California Career Advancement Academy Evaluation, and the Workforce Performance Benchmarking Project. Dr. Freely has recently authored Tuning in to Local Labor Markets: Findings from the Sectoral Employment Impact Study, A Foot in the Door: Using Alternative Staffing Organizations to Open up Opportunities for Disadvantaged Workers and Job Training That Works: Findings from the Sectoral Impact Study.
Prior to joining P/PV, Dr. Freely spent three years managing the Metropolitan Philadelphia Indicators Project (MPIP) at Temple University. At MPIP, he worked with community organizations, government agencies and private sector users to develop research projects using MPIP data to enhance their operational capabilities. Before joining MPIP, Dr. Freely managed a research project funded by the Charles Stewart Mott Foundation that investigated how human resource practices promoted the employment retention of welfare-to-work program participants. Dr. Freely received his Ph.D. in sociology from Temple University.
Online at http://www.ppv.org/ppv/initiative.asp?section_id=29&initiative_id=66 there is a six-minute video of Dr. Freely presenting the findings of the study at a briefing for Congressional staff, staff from the Department of Labor, and other leaders in the workforce development field, sponsored by the office of Senator Arlen Specter. There are actually nine separate videos of different people presenting information about the study. Dr. Freely is the one in the top row on the right with his hand outstretched. Dr. Freely presents the findings from the study using graphs and visual aids. The presentation is also available directly on YouTube at http://www.youtube.com/user/PPVentures#p/u/8/yLUM3GFfQkA.
In recent years in communities all over the United States, there has been considerable experimentation and development of alternative approaches to help low-income people build skills for particular industry sectors. The Sectoral Employment Approach study was a rigorous evaluation of one of these approaches - sector-focused skills training. The study uses an experimental random assignment research design in evaluating three well-implemented, sector-focused training programs to see if these programs make a difference to the earnings of low-income disadvantaged workers and job seekers. In addition, the study identifies common elements to these three programs deemed important to their success.
This discussion was structured around some guiding questions that Dr. Freely focused on each day to provide content for the discussion and needed background information. All of the questions are covered at least in part by the paper that reports on the study, Tuning In to Local Labor Markets: Findings from the Sectoral Employment Impact Study that can be accessed at http://www.ppv.org/ppv/publications/assets/325_publication.pdf.
The Discussion Summary
What defines "sectoral employment"? Describe how it works.
Dr. Freely responds:
Hello everyone, and thank you very much for inviting me to moderate this discussion about Sectoral Employment approaches. It is a topic I am passionate about, and I look forward to a spirited discussion over the next week.
Over the past two decades, an innovative approach to workforce development known as sectoral employment has emerged, resulting in the creation of industry-specific training programs that prepare unemployed and under-skilled workers for skilled positions and connect them with employers seeking to fill such vacancies. The approaches that fall under the name sectoral employment can vary widely and include:
* Skills training programs that provide participants with the skills to enter a particular industry sector or occupation;
* Industry partnerships that identify training needs of incumbent workers in specific industries;
* Organizations focused on systems change, such as efforts to improve wages and working conditions for particular occupations like home health aides; and
Around the country, sectoral programs are being implemented by an array of institutions, including community colleges, Workforce Investment Boards, state agencies and employer associations, with funding from both public and private sources.
In P/PV's recent report, Tuning In to Local Labor Markets, http://www.ppv.org/ppv/publication.asp?search_id=0&publication_id=325§ion_id=0 my colleagues and I presented results from a random assignment study evaluating the effectiveness of three mature, nonprofit-led sector-focused training programs.
For a comprehensive definition of the sectoral approach, I recommend reading Sectoral Strategies for Low-Income Workers: Lessons from the Field http://www.aspeninstitute.org/sites/default/files/content/docs/07-014.PDF , (pages 10-11) by Maureen Conway and her colleagues at the Aspen Institute and Targeting Industries, Training Workers and Improving Opportunities http://www.ppv.org/ppv/publication.asp?section_id=26&search_id=&publication_id=325, (pages 1-2) by my colleagues at P/PV. For the purposes of this discussion, I am going to concentrate on defining the types of sector-focused training programs examined in our study. Each of the three organizations that participated was different—one was a community-based organization http://www.jvs-boston.org/ focused on medical and basic office skills in Boston; another was a social venture http://www.perscholas.org/ focused on information technology in the Bronx; and the third was an employer-union partnership http://www.wrtp.org/ focused on healthcare, manufacturing and construction in Milwaukee. Despite their differences, however, the programs did exhibit certain common elements:
A sector focus: All three study organizations focused on an industry or a small set of industries and have developed industry-specific expertise and relationships that support the training program's design and ongoing adaptation. Among the three sites there was a continuum of strategies to engage industries, including programs that worked with employers one-on-one to understand specific needs or with discrete sets of employers, as well as an employer/union membership association that sought to organize employers from targeted sectors to define common needs.
Integrated skills training: Programs provided training on the range of skills necessary to be successful on the job, including technical job-specific training, job-readiness workshops that were often taught through the lens of a particular industry setting, and support to strengthen basic English and math skills. All three focused on making training accessible and relevant; some provided all components in a single program/package, while others contracted part of the training out to other agencies.
Concern for candidates' career match: All three organizations had recruitment, screening and intake processes aimed at making appropriate career matches for participants. While in some cases programs needed to recruit participants to meet public funding guidelines, all identified candidates with an interest and aptitude for success in their target industry as well as the basic skills needed to benefit from training. Occupation-specific requirements such as driver's license for construction jobs as well as basic skill levels were part of the screening processes. In addition, interviews to assess interest in and commitment to the sector were often an important part of the selection process.
Individualized services to support training completion and success on the job:
In addition to providing a core training program to participants, the three organizations offered a range of social supports, such as childcare, transportation, housing and financial assistance, that met individuals' specific needs. This type of help included assistance to get a driver's license to reach work sites and tutoring to pass a qualifying exam. The programs either provided these services directly or in partnership with outside public or private agencies.
Flexibility to adjust to a changing environment: All three organizations made changes in their programs during the study period. In some cases, their close connection to industry needs led them to shift either occupational or industry focus or to make changes to their curriculum. Programs also altered the mix of services they provided, responding to changes in partner agencies or available funding.
In many ways, I see these programs as being the logical next step for individuals who have been participating in literacy or adult basic education programs or have been stuck in the low-wage labor market. The programs in our study had reading-level requirements that ranged from the 6th grade up to a 10th grade minimum reading level. While some of the programs provided instruction in basic math and literacy, they often referred applicants with more serious reading and math deficits to programs that could focus on meeting these needs-to bring the applicant's basic skills up before entering technical skills training.
With that introduction, I am eager to hear your questions about how we define these programs and how they might benefit the individuals you work with.
Joshua L. Freely, PhD
Senior Research Associate & Director of Labor Market Research
Discussion extension and question posted by Wendy McDowell
Thank you so much for dedicating your time to participate in this discussion. Some of us have been looking forward to this since you were a panelist in Chicago at COABE on transitioning adults to work, training and
postsecondary. First, I could not access Sectoral Strategies for Low-Income Workers: Lessons from the Field from the posting. If others are also experiencing that, it can be accessed at: http://www.aspenwsi.org/publications/07-014.pdf .
It seems to me, whether the organization providing services to low-skilled, low-income individuals is non-profit, union based, community based or privately funded, it is important for the organization to know their
community, both the areas of growth and need. Is it more likely the organization will try to seek an industry that is in need or do the industries seek out the organization and request workers with certain skills?
Dr. Freely responds:
Thank you very much for the welcome and for sending along the correction, I apologize for the bad link.
I think you have hit upon one of the most important aspects of sectoral training and the real key to running a successful program. While these programs can take different forms, as you point out, a common element among successful programs is that they have developed a deep knowledge of the dynamics of their local economy and established strong relationships with employers. It is critical for training programs to understand how their local labor market is shifting, which industries are growing and shrinking to understand where job opportunities will arise in the future. Programs can use their relationships with employers in the selected sectors to anticipate future workforce needs and tailor their skills training to meet those needs.
For example, one of the sites in our study, WRTP in Milwaukee, had provided skills training in both manufacturing and construction for a long time. Over the course of the study period, WRTP observed that the number of job openings in the manufacturing sector was declining significantly as opportunities in this sector were moving to other places around the country and the world. At the same time, WRTP was hearing from its employer partners in the construction sector that there was a growing need for workers with skills handling hazardous materials. Based on this information, WRTP suspended its training in manufacturing and developed a hazardous materials certification course, anticipating the changing needs of local employers and increasing the likelihood that its participants will find opportunities when they have completed the training.
Regarding whether programs seek out industries or industries seek them out, I would say that it is more common for the program to identify an industry that has workforce needs and develop training to specifically address those needs. However, we have seen examples of employers seeking out specific programs, particularly when those programs are well established within a local labor market. Again WRTP is an example. Because it is a Labor-Management partnership, often its employer partners will "order-up" a specific training to meet a particular need.
Joshua L. Freely, PhD
Discussion extension and question posted by Valerie Hunt
Paid internships and customized trainings, wage subsidy programs
I directed a program at a community college partnering with business, DVR and the college to train and employ persons with disabilities in business and financial services. It included a paid internship with the employer paying for the wages for "temporary" status until they completed a successful (3) month internship or what would be the normal probationary period. I would like to see more employer partnerships to customize training programs to fill their needs. There has been interest in the floral design area, custom detailing, windshield repair, etc. for short term training programs. The Workforce Center typically only provides wage subsidy positions in health, construction or green energy, hotel/hospitality, and I forgot the other industry top of my head.
Have any thoughts on this?
Dr. Freely responds:
This sounds like a very interesting program. Did the participants receive any skills training prior to starting the internship? While these types of transitional employment or subsidized work programs are often considered separate from the types of sectoral training programs we studied, I think the combination of the two approaches can be very powerful. In fact, one of the programs in our study, Per Scholas in New York, did have a paid internship component that was available to participants after they completed the 15 week training course. I think that providing participants with an opportunity to gain valuable experience within the sector for which they have received skills training could really increase the likelihood that they will find a permanent foothold in the labor market after the training ends. The occupations you mentioned are also rather unique, I have not encountered many programs that train workers in floral design or auto detailing. Is it your sense that there is a demand for skilled workers in these jobs in your area? I am interested to hear what others might think.
Joshua L. Freely, PhD
Discussion extension posted by Donna Brian
Aspen Institute study
The following very appropriate announcement came to me yesterday from the Aspen Institute. Since it could relate to sectoral training, I'm forwarding it today as a part of the discussion.
Construction is an industry that continues to attract attention as policy makers and workforce leaders seek to connect people to jobs in the current economy. New policies promoting energy efficiency and supporting the development or re-development of our national infrastructure have the potential to create significant demand for construction trades skills. In a new report http://aspenwsi.org/publications/10-014.pdf, Aspen Institute's Workforce Strategies Initiatives (WSI) has sought to investigate how pre-apprenticeship programs can prepare individuals, particularly low-income, minority and women job applicants, to take advantage of emerging career opportunities in construction. Through interviews with leaders of 25 promising and innovative construction pre-apprenticeship programs, WSI explored a variety of factors that impact program design, and identified ways in which the contributions of these programs could be enhanced as part of a broader workforce development strategy for the construction sector. This report was funded by the Annie E. Casey Foundation and is part of a larger project exploring the capacity of the workforce system to prepare individuals for jobs in the construction industry. The project also includes a census and 13 in-depth profiles of construction pre-apprenticeship programs.
One of the findings of the study was:
"Programs often struggle to find resources to build and maintain effective industry partnerships that would connect graduates to jobs and strain to support program graduates for an appropriate amount of time after job placement."
That seems to be one of the main strengths of the training partnerships featured in the study that P/PV did: there was a strong connection or partnership between those providing the training program and the industry the participants were being trained for. Relating to this finding, one of the recommendations of the study is:
"Above all, we found that pre-apprenticeship programs play a significant role in developing a skilled and diverse construction workforce. This role could be expanded, however, through additional support to local
programs. In particular, the following support could help build and sustain program capacity:
* Consistent funding to maintain program capacity, and, in particular, to support the development of industry relationships and the provision of post-completion services to trainees, would help programs better serve both workers and employers. Access to WIA funding in particular could be improved through guidance that considers some of the specific needs of the construction sector."
(Both of the above quotes are from the introduction to the study.)
Maureen Conway of Aspen Institute’s Work Strategies Initiative responds:
I'm Maureen Conway and I worked with Josh on the impact study report and with my colleagues here at The Aspen Institute on the report cited below--I'm so pleased to see you found the construction report of interest too! I will offer a few comments on the highlights you noted, particularly as they relate to sector programs in general and the participants in the impact study in particular. And I'll throw in a quick response on the question posted yesterday.
First, with respect to building and maintaining relationships with industry, I would say that this is an issue that is a work-in-progress for nearly all sector programs. The programs in the impact study have made a lot of progress in this area to be sure, as have the programs operating construction training programs that we interviewed. But engaging industry is something that leaders of sector programs are always attentive to and seeking to improve--whether they're trying to expand and improve market share, deepen their relationships so that they can touch more occupational levels in an industry, or simply maintain relationships in the face of staff turnover at partner employers or within their own agency or in the context of little demand for new employees. This aspect of the work requires constant attention. However, this constant attention is often not well supported through traditional funding streams. Many of the activities that program staff engage in to build and maintain employer relationships--such as going out and meeting with employers at their worksite, becoming involved in a local trade or business association, and other activities, will fall outside the list of activities to be supported in a training implementation grant or contract. In addition, hiring staff who have experience in the target industry has been a particularly powerful way for a program to enhance its understanding of industry skills/norms and, critically, to expand its industry network--but people with industry experience often have higher salary expectations than what can be supported with typical funding streams. So these are some of the challenges that the construction programs we spoke with face, and that many other sector program leaders face.
My second comment relates to post-placement support. Like most industries, construction has its own particularities of how work is organized that should, and often do, influence program design. One of the features of many construction labor markets is that workers often move from job to job and experience periods of unemployment between jobs. Learning how to navigate that environment in order to maximize work time and manage income during the down times is an area where many new construction employees need post-training help. While many programs work to prepare their trainees for this fact of the industry during training, sometimes the reality of what that will mean doesn't really sink in until students experience it, and post-training support can be critical at that time. Again, this was an area programs reported was not typically well supported.
With relation to both developing industry relationships and post-program support services, in other parts of the paper we also called for attention to capacity building and leveraging the experience of the field. As more and more organizations adopt a sector-oriented approach to their work, it would be terrific if we could find new ways to accelerate the learning curve and support program staff development.
Before I sign off, I also wanted to make a quick comment to amplify some of what Josh said yesterday in a response to a question about how programs find their way into particular sectors and connect with specific employers. I think the programs in the study in many ways exemplify three different avenues that I've seen. One starts with looking at what businesses are important in our regional economy, another starts with a particular set of workers and considers what would be good opportunities for them, and a third builds on existing organizational knowledge and relationships within an industry. WRTP began with manufacturing as it was experiencing the loss of manufacturing jobs and consequently the loss of many of the higher paying jobs for its residents. And it began by building relationships with employers and unions, with the angle of helping the workforce be competitive in order to hang on to manufacturing jobs. From that vantage point, they then added construction trades and developed a skilled trades Center of Excellence. The organization works with community based nonprofits to identify candidates that are ready for their programs and in need of better jobs. JVS Boston, on the other hand, has its roots in service to the low-income job seeker, and began more with the question of where there are good opportunities in the labor market for the people they are trying to help find better jobs. And Per Scholas has the "related business" approach to choosing a sector. Already having a computer refurbishing enterprise, the organization had some assets in terms of industry connections and knowledge, and from there saw an opportunity to build out a training approach that could enhance its service to the community. While the programs do indeed generally go out and find the employers, which employers they go to and why may vary. But all of them can work.
OK, that's enough from me. I look forward to following the discussion on the list.
Director, Workforce Strategies Initiative
The Aspen Institute
Tell us about the study you did:
* What were the research questions?
* How was the study designed?
* What/Who were the programs/participants studied?
Dr. Freely responds:
Today's discussion questions involve the design of our study: Tell us about the study you did: What were the research questions? How was the study designed? What/Who were the programs/participants studied? I will paste detailed responses to these questions below, drawing mainly from the report but I thought I'd highlight a few important things first.
* The primary question this study sought to answer was: Do mature sector-focused programs result in significant labor market gains for low-income, disadvantaged workers and job seekers?
* We used a rigorous random control trial design to answer this question. Applicants went through the entire application process and after they were determined to be eligible to enter a program, they were randomly assigned into a treatment group that received services from the program or a control group that received no services (though controls could find services at other programs not in the study). This design is important because one explanation for good outcomes observed in sector programs in the past was that they creamed, only picking the best candidates. This design randomized study participants after the selection process was completed, or after the so-called creaming occurred.
* Three sites were selected for the study and there are detailed descriptions of each below. While they were quite different from each other, each had been doing workforce development for a number of years, each had the capacity to recruit a large number of participants into the study, and each was running programs that in some way matched the criteria I outlined in my post yesterday identifying them as sector programs.
* The makeup of the study participants is discussed in detail below. It is clear that, while they are not the hardest to serve individuals, most had significant barriers to employment. Though most worked during the year prior to entering the training program, they did not work consistently and would be considered low-income by any measure.
P/PV used an experimental research design to bring as much rigor as possible to the following question: Do mature sector-focused programs result in significant labor market gains for low-income, disadvantaged workers and job seekers? More specifically, we strived to determine whether such programs raise the earnings of program participants and whether participants were more likely to find employment and work more consistently. We also wanted to explore whether program participants obtained higher-quality jobs. For example, were participants more likely to earn higher wages? Did participants find jobs with better access to benefits? Further, we set out to explore whether specific groups of people, such as welfare recipients or young adults, benefit from participation. We also sought to understand the programmatic, contextual and individual factors that contribute to these outcomes.
To answer these questions, 1,286 people were recruited for the study from across the three programs over a two-year period, all of whom had been through their program's application process and met its eligibility criteria. Baseline data were gathered from eligible applicants through a phone survey about their education and work histories, additional sources of income, living situations and experiences with other employment programs. Then, half of the participants were selected at random to participate in the program (the treatment group); the remaining half (the control group) could not receive services from the study sites for the next 24 months, but they were free to attend other employment programs or seek access to other services. No significant differences existed between the treatment and control groups at the time of the baseline survey).
Members of both groups were surveyed by phone between the 24th and 30th months after the baseline survey was conducted. During the follow-up survey, participants were asked to provide detailed information about every job they had worked during the study period, including earnings, months worked and weekly hours, and whether participants were offered and had taken advantage of benefits. The follow-up survey sample included 1,014 respondents, reflecting a 79 percent response rate (75 percent for the control group and 82 percent for the treatment group). The programs' effects were measured by comparing the progress made by members of the treatment group with that made by members of the control group. Because assignment to these groups was random, any differences found between treatments (hereafter referred to as program participants) and controls can be attributed to participation in the sector-focused training programs.
In addition to collecting data about individuals, we conducted regular site visits to each of the three organizations. The goal of this qualitative research was to identify key practices as well as challenges the organizations faced. Once a year, P/PV interviewed both frontline staff (such as job developers, case managers and career specialists) and supervisors and senior management. Focus groups were also held annually with participants, and on occasion interviews were conducted with employers and board members of the participating organizations. Although the study design did not include the collection of detailed information on program intensity or the costs associated with program implementation, the qualitative component of our research enabled us to document the structure and content of the programs.
We did not seek organizations that followed a common model to participate in the study, as sectoral programs employ various approaches depending on the organization leading the effort and local employers' needs. Instead, we sought mature programs that seemed to be well implemented, since it takes time for an organization to both understand employers' needs and craft appropriate responses.
Through nominations from leaders in the workforce development field, P/PV identified 25 organizations that targeted an occupation or cluster of occupations, that aimed to place participants in jobs paying $8 or more per hour, that served more than 100 participants annually and that had been operating their programs for at least three years.
Three organizations were selected to participate in the study:
* The Wisconsin Regional Training Partnership (WRTP) is an association of employers and unions that seeks to retain and attract high-wage jobs in Milwaukee and create career opportunities for low-income and unemployed community residents. WRTP develops training programs (generally lasting between two and eight weeks) in response to specific employers' requests or to clearly identified labor market needs. Its short-term pre-employment training programs in the construction, manufacturing and healthcare sectors were included in the study. Study participants were primarily African-American, about evenly divided between men and women, and about 40 percent had been incarcerated.
* Jewish Vocational Service-Boston (JVS-Boston) is a community-based nonprofit that has provided workforce development services for more than 70 years, including operating one of three One-Stop Career Centers (One-Stops) funded by the Workforce Investment Act (WIA) in the Boston area. The organization aims to serve a diverse range of Boston's disadvantaged populations, including refugees, immigrants and welfare recipients. Its training programs in medical billing and accounting were included in the study. Each training program was provided over 20-22 weeks for 20-25 hours per week. JVS engaged its target industries by forming employer advisory committees and building individual relationships with local businesses. JVS-Boston study participants were primarily women and included a large number of young adults and current or former welfare recipients.
* Per Scholas is a social venture in New York City that combines a training program with efforts to refurbish and recycle "end of life" computers and distribute them to low-income people through partnerships with nonprofits, schools and community colleges. Per Scholas' computer technician training program-which prepares participants for jobs in repair and maintenance of PCs, and printers and copiers, as well as the installation and troubleshooting of computer networks-was included in the study. The training program consists of 500 hours over 15 weeks and is aligned closely with the A+ certification exam-an industry recognized computer technician competency assessment. Program participants also take part in internships, during which time they work in the Per Scholas recycling and refurbishing center or with local employers around the city. At Per Scholas, study participants were primarily male, and a sizeable proportion was foreign-born.
Participants in the study were screened through their respective programs to ensure they had the basic academic skills to read and understand instructional material; entrance requirements ranged from 6th to 10th grade reading and/or math levels. In the year prior to the study, participants had been in and out of the labor market. Only 10 percent had worked full-time for the entire year, and the average participant had worked full-time for three and a half months. Thirty-four percent were working at the time they enrolled in the study. On average, each had worked (for at least one hour) in seven months of the year prior to the baseline survey, earning $9,872. Nearly 40 percent had received public assistance at some time, including the 23 percent of participants who were on welfare at the time of enrollment.
Programs enrolled a sizable number of young people. Since definitions of youth and young adults vary among practitioners, researchers and funders, we analyzed the data according to two groupings: ages 18 to 24 and ages 18 to 26. Twenty-eight percent were under the age of 24, while 37 percent were younger than 26. The average age was 32. About one in five participants had been convicted of a crime. Seven percent had been homeless in the year before the baseline survey was conducted. In terms of their educational credentials, 53 percent had only a high school diploma, 22 percent had a GED and 18 percent had more than a high school diploma (an associate's, bachelor's or master's degree). Overall, women and men were almost equally represented in the study sample, though there were differences across sites. We conducted analyses on several of these subpopulations at each site, though such analyses were sometimes limited due to small sample sizes.
Joshua L. Freely, PhD
Discussion extension and question posted by Valerie Hunt
I am doing a blackboard class, so can't participate as fully as I'd like. USDOL funds have always been available for on the job trainings, work experiences, etc. Now we are calling them transitional jobs or wage subsidy positions; semantics. I believe we must meet employers’ needs and though these positions may not be in occupational job growth projections, working with your local employers brings opportunities for many customized training opportunities such as auto detailing, floral design, etc. I ran a program to train persons with disabilities as bank tellers with banks and credit unions and that was a partnership through employers, DVR and community college. More collaborations including the employer create more commitment and partnerships. How do sectoral employment programs differ?
What did the study find? Conclusions? Implications?
Dr. Freely responds:
Today's discussion question is about the findings of our study. Rather than reproducing all of the results outlined in our report, I thought I would just summarize some important points below. It is important to note that the random assignment nature of the study allows us to attribute these findings directly to the impact of participating in sector focused training. When compared to their control group counterparts, participants in the sector focused training programs:
* Had significantly higher earnings - about $4500 or 18% more over the two year study period and $4000 or 29% more in the second year of the follow-up period. In the second year, program participants learned an average of $17,500 while control group members earned approximately $13,500.
* Were significantly more likely to work and worked more months. Over the two year follow-up period, program participants worked about 1.5 more months than controls. By the second year of the follow-up, about 70% of program participants were working in an average month versus about 60% of the control group members. Program participants were significantly more likely than controls to work all 12 months in the second year (52 percent versus 41 percent)-an indication that sector-focused training programs helped participants find steadier employment. Finally, program participants worked significantly more hours, 250 more or over 20 more hours a month in the second year of the follow-up.
* Were more likely to work in jobs that paid higher wages. Specifically, they were more likely to work in jobs that paid at least $11 an hour AND in jobs that paid at least $13 an hour.
* They were more likely to work in jobs that offered benefits (paid sick leave, paid vacation, health insurance, or tuition reimbursement) and worked more months in these jobs than controls.
* When each site was examined individually, program participants at all three had significant earnings and employment impacts though the size and timing of those impacts varied and often reflected the different approaches that each program took toward training. For instance, participants at WRTP, where the length of training was the shortest, saw higher earnings over controls as early as the second month of the follow-up period. Participants at Per Scholas, where training was longer term and included the option for a post-training internship, did not see significantly higher earning than controls until well into the second year of the follow-up period, however these differences were particularly large and resulted in significantly higher earnings for Per Scholas participants in the second year as a whole.
* Finally, we looked into the impact of these programs across a wide range of sub-populations (men, women, young adults, the formerly incarcerated, and several others) and found significant impacts for these participants, though these effects varied from site to site. This variation reflects differences in the populations served at each site, their different sector concentrations, and the variation in their approaches to training. The findings of this study led us to several conclusions:
1) Mature, nonprofit-led sector-focused programs can increase the earnings of disadvantaged populations.
2) Variation in approaches can be effective but results in different effects on earnings.
3) Mature, nonprofit-led sector-focused programs can be effective with a range of disadvantaged workers and job seekers.
4) Nonprofit organizations can play a critical role in delivering workforce services. The three programs in this study demonstrated an adaptability that allowed them to connect disadvantaged job seekers to employers using a mix of strategies and a range of public and private funding sources.
I am eager to hear the list's reactions to these findings and would be happy to answer any questions you might have about them. Thank you to everyone who has responded so far, I appreciate the feedback and the dialogue.
Joshua L. Freely, PhD
Discussion extension and question posted by Stephanie Moran
We have started a "Developing Career Pathways" course for students in our GED classes who might also be at the ABE level but want to learn more about how to land, keep, or progress in a job/career. I would love eventually to tailor it/tie it to a particular field such as allied health but am keeping it general for at least two terms (about 8 weeks each). Because we are an adult ed center, not a community college, our classes are relatively small (under 15 Ss, sometimes fewer), so to narrow the course's focus might exclude some students. Anyone who is in a similar situation but further along in your approach/curriculum, I'd appreciate hearing how you achieve good results and how you track your results.
I'd also appreciate hearing from people who have used Key Train and WorkKeys
Discussion extension and question posted by Wendy McDowell
It is very interesting to me that the significantly higher wages were stronger in the second year. Do you know the time frame participants were eligible for support services once employed? Was there any correlation
between the length of time support services were in place and the job retention rate?
I would like to hear about some of the methods used by sectoral programs to track participants for longer than six months.
Thanks Josh for this great summary.
Dr. Freely responds:
Great questions. Part of the explanation for why earnings were higher in the second year was that program participants spent at least part of the first year in training and not working (while many controls went out and found employment). So, the fact that we saw two year earnings differences is important when you consider that some program participants spent as much as six months of the first year of the follow-up period in a training program. We looked at the second year of the follow-up by itself to give us a sense of the impact these programs might have had after the training was completed and the participants had time to establish themselves in the labor market. It is encouraging that the impact of these programs is still observed, and may even strengthen, several months after the training has ended.
That said, each of these programs had intensive post-placement follow-up services available to participants for the entire two years of the follow-up period. Each program had dedicated retention specialists and in some cases, participants were contacted on a monthly basis for each of the 24 months. Retention staff could help participants overcome problems that might impact their success on the job or help re-place participants who found themselves out of work. While we can't compute the direct effect of these post-placement services, I would venture that they certainly contributed to the success of the sectoral trainees.
While the programs stayed in contact with many of their participants for the up to two years, P/PV used an outside survey firm to track and interview all participants (and controls) to collect information for the study. I hope this answer provides some clarification.
Joshua L. Freely, PhD
Discussion extension and question posted by Blaire Willson Toso
Your study is both fascinating and encouraging. From what I have read the support services are incredibly important in helping people to get, keep, and be promoted in a work setting. I think this is related to the idea that life skills are also a part of the workplace culture. Did your study address this at all? Or, do you have any thoughts on this?
Were immigrants part of the study population? If so, how did language factor into the training or was there an assumed proficiency level in order to be a part of the training?
What do you think it would take to get people who go through this training to continue their upward economic momentum? Or, do you think that this type of program serves to give participants enough of a skill base and an understanding of how the world of work functions to navigate it on their own?
I realize that two of these questions are a bit unwieldy and speculative, but take it as a compliment, this discussion has gotten my wheels turning.
Dr. Freely responds:
Thank you for the excellent questions. I’ll try and address each one individually.
1) I think you’ve identified one of the key findings of this report, that technical skills alone may not equal success. My colleague and co-author Sheila Maguire likes to refer to the types of programs we studied as “training-plus” programs. All three integrated job readiness and basic skills components into their curricula to address some of the softer skill deficiencies they often observed in program participants. This might include instruction on proper interviewing and personal presentation techniques, workplace culture, and workplace literacy and math courses that contextualize basic skills instruction to specific occupations. In addition, each program offered supportive services to help participants overcome some of their barriers to success in the workplace. For example, programs might offer bus passes to participants without vehicles or connect them with affordable childcare providers. If participants lacked proper clothes to wear on interviews, programs could refer them to services that provide free clothing. At each site, case managers assessed the needs of each participant and tailored a service plan to meet those needs. Unfortunately, the design of our study limited our ability to make an empirical link between the provision of these services and participant success, however we are continuing to work with the data and hope to release some additional reports aimed at addressing these issues for practitioners.
2) Immigrants were a part of the study population though only at two of the sites – JVS-Boston where they were about 40% of the study population and Per Scholas where they represented about 25% of the study population. The two sites differed in their requirements for language proficiency. Per Scholas required that all of its applicants were proficient in reading English at a 10th grade level and as a result, immigrant English proficiency was quite high (immigrant participants at Per Scholas were also more highly educated, in general, than participants as a whole). JVS-Boston’s literacy and language requirements were significantly lower and they offered ESL classes as a part of their regular training curriculum for those who needed it. It is worth noting that across the entire study sample, immigrant participants did quite well when compared to immigrant controls. This was particularly the case at Per Scholas where the earnings gains for immigrant participants were among the highest in the study. Immigrants at JVS-Boston did not see gains that were as strong.
3) I think this question is really key but it is a little harder to answer from the information gathered in our study. Theoretically, the skills obtained through sector focused training programs prepare an individual to enter a career pathway, not simply a job. After training, the participant enters a job that is like a rung on a ladder, after a period of gaining experience and perhaps more skills, the individual advances up the ladder to more lucrative employment. I would especially invite my colleague Maureen Conway at the Aspen Institute to weigh in on this because her work has really contributed to our understanding of this. Because our study only followed study participants for 24 months after they entered training, we are only observing the beginning of this process. While our study provides some interesting clues as to how it works, particularly in certain sectors (the findings at Per Scholas in the IT Sector come immediately to mind), we would really need to follow participants for several more years to fully understand if they are advancing in the way the theory suggests. While we don’t currently have plans for a longer follow-up, we are exploring ways we might dig further into this issue.
Thanks again for the great questions, I hope my answers at least kept your wheels turning. Please keep them coming. In the meantime, I will get work answering today’s discussion questions.
Joshua L. Freely, PhD
Maureen Conway responds:
I think your question about whether the training is sufficient to maintain upward momentum or not is an excellent question, and one that is in need of further research. We have done a little research in this vein, albeit of a more qualitative nature, that you can see in a report we released in 2006, entitled Skills to Live By. You can access the report here: http://www.aspenwsi.org/publications/06-010.pdf In this work we did a series of focus groups and individual interviews with people who had participated in sectoral training programs four years previously, in which former trainees reflected on what elements of the training had helped them most in their career, what they perceived as facilitating further advancement, and what were continued challenges. In general, participants felt the things they learned from the training continued to help them with their careers and that the thoughtful design of the program they had participated in, which considered their life situation as well as their skill needs, was critical to their continued participation and learning in the program. However, the pathways individuals followed post-training were greatly influenced by their personal/family situation and how that evolved as well as the sector/occupation they gained employment in and the type of opportunities it presented.
I would also speculate that the strength of the labor market someone is working in would have a profound outcome on whether the success an individual has achieved can be sustained, and that the level of help and support that is needed in a tight labor market to sustain continued advancement might be different than what is needed to sustain career growth in a more challenging economic environment.
Discussion extension and question posted by Blaire Willson Toso
Josh, thanks for your answers.
I am not surprised to learn of the immigrant outcomes. It is great to learn of programs that are successfully working with higher educated immigrants. As you probably know, appropriate work opportunities for highly educated immigrants can be difficult to find.
Does anyone on the list know of other types of programs that are working to certify or assist immigrants with professional degrees to transition into jobs that acknowledge their home education or experience?
One more quick question - how were these programs funded? Particularly pieces like the case managers, etc. This could be helpful for those who would like to develop or run a program that provides this kind of support but need the funding to do so.
Dr. Freely responds:
The funding of these programs is as varied as their approaches. One site, Per Scholas, was funded almost entirely through private foundation support during the study. The other two sites were funded through a mix of public and private dollars. Perhaps the biggest challenge faced by all of the programs in the study was cobbling together the various funding streams that support workforce development to bring the necessary resources to the program. This might include using WIA-ITA money, TANF training grants, and various grants from private foundations to fund the different components of the training. Each funding stream also comes with reporting requirements, restrictions on how funds could be used, even their own databases that needed to be maintained and negotiating these conditions represented a drain on the capacity of the organization. Time spent entering data into 4 different funder databases is time not spent providing services or conducting training. One of the policy recommendations that we have made out of this study is streamlining the funding around workforce development to reduce the burden of fundraising for programs such as these.
Joshua L. Freely, PhD
Discussion extension posted by Miriam Burt
In response to Blaire’s question about other programs that assist highly education/skilled/professional immigrants to become recertified and to get jobs in the U.S. that are commensurate with their skills and experience, there is Upwardly Global.
Here’s their website. http://www.upwardlyglobal.org/
I’ve heard the director/founder Susan Liu speak at conferences and it seems a key factor in this endeavor is that upwardly global works both with the prospective employers and the jobseekers.
Discussion extension posted by Donna Miller-Parker
I would like also to recommend the work of the Welcome Back Centers around the country (for those fortunate enough to be located near one) They do outstanding work with re-certifying and upgrading training for immigrants experienced in health care fields.
How should the results of the study guide further practice?
Will follow-up continue with these programs/participants?
Dr. Freely responds:
Today's discussion questions are: How should the results of the study guide further practice? Will follow-up continue with these programs/participants?
The results of this study demonstrate that well-run sector focused training programs run by mature non-profits can make a significant impact on the lives of their participants. In my post on Monday morning I outlined the 5 elements that we see as being essential to the success of sector programs. A challenge in replicating or scaling this approach is that it is inherently local, deeply connected to the local labor market and the needs of employers and job seekers in a particular area. As Maureen noted in her earlier post, the choice of which industries and which populations to serve is often dictated by the local context. For this reason, the implementation of the key elements for successful programs may differ from program to program. For instance, a program focusing on the IT industry in New York (like Per Scholas) will recruit from a very different population than a program that serves the healthcare industry in Boston (JVS-Boston). The choice of sector influences the recruitment and screening of applicants, which in turn determines the mix of services and job-readiness training the program must offer. On the other hand, a program that is committed to serving a specific population, perhaps the formerly incarcerated, must seek out industry sectors that will provide participants with opportunities. Often, employers in the healthcare and childcare industries won't hire people with criminal records so training programs that serve large numbers of ex-offenders might provide training in construction and manufacturing instead. Having the capacity to recognize the needs of both employers and job seekers within the local labor market and having the flexibility to adjust the approach to meet those needs is, I think, of primary importance for organizations that want to run successful sector focused programs.
Having completed the report on the experimental findings from this study, my colleagues and I at P/PV are turning our attention toward using the data to provide useful lessons that will guide practice and improve performance. These reports will be released intermittently over the coming year and I will be sure to inform the list when they are released.
As I stated in my post earlier this morning, we don't currently have plans to follow-up further with program participants, the cost and logistics of fielding another survey are prohibitive. That said, we are in the beginning stages of exploring alternative ways to gather information on study participants for a longer term follow-up (such as using administrative data) but there are significant methodological and logistical issues that need to be figured out before we pursue them. We will continue to be in contact with the study sites as our sectoral research work continues.
I am looking forward to hearing the list's thoughts about how these results can guide further practice. I am particularly interested in hearing what kinds of additional information would be most useful in informing list member's work going forward.
Joshua L. Freely, PhD
Discussion question posted by Ilona Kosova
Thank you for this interesting discussion. Joshua, in the course of this study, have you learned anything about employer perspective on training for immigrant employees? I am specifically interested in the balance between EAL and literacy and essential skills training that employers provide for immigrants. How would you encourage employers to see literacy and essentials skills training as a separate issue from EAL?
Dr. Freely responds:
This is a great question, but unfortunately, not one I have a good answer to. Integrating employer perspectives is essential to better understanding the role sector programs can play in workforce development but it just was not a part of our study. While we did conduct interviews with some employers, I am not aware of us collecting information specific to the question you ask. We would like to focus more on the demand side of the sector training equation in our future research and your question will certainly inform that work. I am hoping that someone else on the list might have a better answer. Thanks for the question and I apologize that I can't be more helpful.
Joshua L. Freely, PhD
Discussion extension posted by Paul Jurmo
I'm a big fan of the P/PV and Aspen Institute reports mentioned here in the past week, along with similar reports from CLASP and others who have been making the case for career-pathway/sector-focused systems. This idea of integrating basic education with workforce development services and tying them to particular needs and opportunities in the local labor market has been around since at least the late 1990s when policy makers and others were arguing for this idea. However, these recent reports are providing more details about how to implement this idea.
In the past two years in a community college in NJ, we tried to implement these ideas in projects geared to the eldercare, transportation/ logistics/distribution, and other industries. We created curricula which helped job seekers understand job opportunities in selected industries, develop career plans, develop various kinds of work-readiness (EFF) skills useful in those industries, and develop particular kinds of technical knowledge and skills (e.g., small business skills for truck drivers, basic electronics skills for maintaining equipment used in warehouses) needed for selected jobs in those industries. As the P/PV and other reports suggest, we also made special efforts to carefully select learners (so they understood what they were getting into and were committed to doing the work required), get local employers involved (in shaping curricula, serving as guest speakers, reviewing graduates' resumes), select and prepare instructors to do this kind of specialized program, provide academic and job counseling to help learners connect to ongoing opportunities, and ensure that facilities were well-equipped with computers (so learners could develop both computer skills and other skills simultaneously and be prepared to continue their learning after the courses were over).
Though this took a lot of work by a well-organized team, I am absolutely convinced that this kind of approach is do-able, effective, and worth developing further. The attached report from one of our first such initiatives (to help English language learners prepare for eldercare jobs) describes how we tried out these ideas. We then adapted lessons from this initial project as we moved into a yearlong TLD (transportation/logistics/distribution) industry initiative. More recently, we adapted elements of this for a program for low-income youth.
A major challenge to moving in this direction, of course, is funding. But it's not just a question of "new funding" but how to use existing funding to support such initiatives. For this to change, all of us -- funders, practitioners, and decision makers at all levels of the system -- need to educate ourselves about these models and be willing to give up old ways of thinking. For that to happen, informed, committed, and courageous leadership is needed.
Dr. Freely responds:
These are very insightful comments from someone who has implemented a sector program. I could not have said it better myself and I won’t try. Thank you.
Joshua L. Freely, PhD
Discussion extension posted by Gary Bartolina
I always look forward to your postings, thank you for the information. ESOL, and other Basic Skills Work related curriculum, is exactly what we are gearing towards in our planning.
Are there other employment initiatives applying study results to their programs?
Dr. Freely responds:
I want to thank everyone on the list for allowing me to discuss our report over the last week, I have found your questions and comments to be incredibly enlightening and helpful. I especially want to thank Donna Brian for arranging and moderating the discussion. I hope that you all found this discussion useful and I look to forward to being an active member of the list in the future. The final discussion question asks: Are there other employment initiatives applying study results to their programs?
Because the final report from the study was only released in late July, it is hard to know how much it has informed practice or policy. We do know that portions of the study were read into the congressional record as part of the debate over the recently passed S.E.C.T.O.R.S Act and we are encouraged that our work may have played a small part in the passage of this important legislation. Over the past 12 months, we have presented the findings and recommendations to practitioners, policymakers, and researchers at various conferences and meetings such as the National Network of Sector Partners, The National Conference of Mayors, Goodwill Industries International's spring learning event, and the USDOL-ETA Recovery Research Conference. We will continue to disseminate these findings to as wide an audience as possible and plan on conducting further research using the study's data to further inform practice and policy.
On this final day of the discussion, I am eager to hear your suggestions for ways we can effectively communicate these findings so that they may help inform and improve practice in the workforce development field. Of course, I would also be happy to answer any lingering questions that you may have regarding our study or its findings.
Joshua L. Freely, PhD
Wrap-Up of the Sectoral Employment Impact Study Discussion from the list moderator
I want to extend to Josh Freely our sincere thanks for enlightening us about sectoral training in general and the P/PV study in particular.
The report that our discussion has been based on has been reviewed and accepted for inclusion in LINCS' Workforce Competitiveness Resource Collection (see the LINCS resource collection at http://www.nifl.gov/lincs/resourcecollections/RC_workforce.html). Here is the information it reports:
Job Training that Works: Findings from the Sectoral Employment Impact Study
What the experts say: This detailed research is the promised follow-up to the brief article published by Public/Private Ventures last year, "Job Training that Works: Findings from the Sectoral Employment Impact Studyhttp://www.ppv.org/ppv/publications/assets/294_publication.pdf. The research study tests the approach of sectoral training in which basic skills, job readiness, and technical training are combined into one workforce training program focused on a particular sector such as construction, medical billing, or computer repair.
Three very different non-profit organizations providing sectoral training in three different locations agreed to randomly assign their accepted applicants into two groups: one would receive sectoral training and one would not. The variation in approach and length of time of the three programs allows educators to see models of how their own new programs might be developed. Public/Private Ventures, with funding from the Mott Foundation, then analyzed the two groups as well as many subgroups over a two-year period to determine the advantages of sectoral training. Although the study does not compare sectoral training with generic literacy or workforce training, it does provide detailed information on the impact of sectoral training.
In addition to reporting the results of the experimental research study, it also included qualitative data about the three non-profit organizations providing the training. Among the aspects discussed were the flexibility of the organizations in changing their programs to meet participants' and employers' need, provision of support for the trainee, relatively long-term and intensive training programs, design of their training programs so that completers achieve external qualifications and certifications, and close working relationships with employers and unions.
The report provides an excellent executive summary for those who do not want to read the entire research report. Since this is a research report, professional development programs on how to provide sectoral training would have to be found elsewhere. The report suggests follow-up research directions for researchers. The endnotes provide a useful bibliography of references related to the study.
The theoretical base is functional context education. Thomas G. Sticht has written most extensively about the effectiveness of combining literacy and workforce training* which results in better general literacy levels than generic literacy instruction. This study tests the effectiveness of sector training or functional context education.
*Sticht, Thomas G. (2003). Functional context Education (FCE) Part 1: New Interest in FCE Theory and Principles for Integrating Basic Skills and Work Skills. http://www.nald.ca/library/research/sticht/oct03/fce/fce.htm
Here's what I would like to know, now that the formal part of the discussion has concluded: Are there those of you who have already implemented, or are thinking of implementing, a sectoral training program who would like to expand on this discussion and more fully discuss your particular ideas with more specifics? If this sounds like an idea worth pursuing to you, please reply to me off-list at djgbrian at utk.edu and I will convene a smaller support group to work together through the practical application of sectoral training to your work. Once I have the names of anyone interested, we will decide on the term limits for the group and what kind of contact to plan. I have several possibilities in mind.
Donna Brian, moderator
LINCS Workforce Competitiveness Discussion List
Please note: We do not control and cannot guarantee the relevance, timeliness, or accuracy of the materials provided by other agencies or organizations via links off-site, nor do we endorse other agencies or organizations, their views, products or services.