Frequently Asked Questions?
What does it mean that the ERS has been validated for repeat administration?
We have designed the ERS so that it can be given up to six (or 12) times to a client in order to measure change over time. Part of our field test was focused on demonstrating that we could get a statistically reliable measure of change when the ERS was repeated.
Are there any clients for whom the ERS is not appropriate?
We found that the only clients for which the ERS was not appropriate were those with developmental disabilities (the conceptual framework was too complex) and those who were actively very manic, very depressed, or very paranoid and not controlled on medication (and so not able to be realistic about themselves).
Were all client groups represented in the field testing?
We tested clients from 14-65 years of age, with representation from Indigenous communities, all disability groups, a range of ethnic backgrounds, and different types of employment history. The specific breakdown of the 758 clients in the field testing is given in “ERS Research Results.”
How can we know how to help a client if we don’t know the types of questions that were asked to determine self-sufficiency?
You can always tell from the feedback report which factors are important for a client to address to become self-sufficient. We provide a section on the ERS Agency site called “Helping Clients Get Employment Ready” where we give factor definitions and sample interventions that would be helpful. In the ERS training, we teach staff ways of exploring with clients what particular aspects of a factor they need help with and then how to review sample sources of help available to them locally, in accordance with these needs.
Is it possible to get a listing of the 75 questions and to link those questions to the various factors?
No, we do not publish the questions or the links between questions and factors. In fact, the ERS license states specifically that users will not “print screen” questions. It would jeopardize the validity of the scale, and particularly the ability to re-test clients and measure change, if we published the link between questions and factors.
Why do you include “Soft Skills” in your Employment Readiness Model™?
We found in the research that some clients with many barriers were still successful while others with few barriers were not. The difference lay in how strong the clients were on the five Soft Skills factors. So we measure Soft Skills both in order to determine how well clients will be able to deal with their challenges and to determine how effective they will be in their work life.
How did you determine the three levels of readiness – Not Ready, Minimally Ready, Fully Ready?
The categories are from the field testing. We analyzed the various configurations that increase the client’s likelihood of success in order to develop the underlying categories.
How did you measure barriers to employment?
We chose to measure barriers or challenges as sources of stress. We identified the 30 most common challenges reported in the international research literature, and then had a panel of experts rate them with regard to their relative stressfulness so that they could be weighted appropriately in the overall ERS scoring.
How did you validate the Employment Readiness Scale™ itself?
The Scale was field tested with 758 clients on employment insurance or income assistance over a period of three years. The reliability and validity of the ERS were tested several ways – e.g., staff ratings of clients were compared with client scores, client scores post-intervention were compared with their initial scores to see if change could be detected, client employment status was compared with clients’ final ERS scores. Please see “ERS Research Results” for more details.
How did you develop and validate the Employment Readiness Model™?
Under contract to the Canadian federal government, we reviewed all available research on barriers to employment and what helps clients become successful in their work life. We looked particularly at research from Australia, Canada, Sweden, the U.K., and the U.S. We then conducted focus groups and interviews with subject matter experts and key informants. The model was ultimately reviewed and approved by a federal-provincial Steering Committee of professionals in employment counselling.
How effective are the questions in predicting clients' work life success?
This can only be determined through thorough, rigorous field testing. This is what we did when developing the ERS. The ERS questions were carefully selected from over 400 initial questions, based on their predictive value. If you were to read through them, some would be obvious choices and you know that they will work (known as face validity). But at least half of them would not be obvious. You might wonder as to why ask that question? The reason is that each question has been tested to have strong predictive power. Alternatively, you may wonder, why isn’t question X not being asked? The reason is that the question tests as having little predictive power. A better way to know whether the Scale will work with a particular client group might be to review our Summary of Research Results.
In brief, our findings have been that the ERS works well with clients who:
- Are at least 14 years old.
- Are from all ethnic and cultural backgrounds.
- Are From any linguistic group.
- With all levels of literacy (some might require assistance if reading levels are low - the reading level of ERS questions is about Year 7)
- With all levels of ability except for those with developmental disabilities (i.e., under 80 IQ) or those with psychiatric disabilities (who are not yet stabilized on medication).
What if I would like to try taking the scale to see if it is valid for my clients?
Simply taking the Scale will not give you that answer. Our three years of field testing were designed to answer that question, and the Scale’s validity was successfully demonstrated with a wide range of clients.
Can we get the raw data in ASCII format for analysis?
No, we do not provide raw data as, for analytical purposes, the data need to be normalized. We provide a variety of ways to code clients so that you can use the Agency site reports to answer questions.
Why would we purchase multiple Agency licenses for different offices rather than using Client Coding?
If you have offices serving communities with different demographics, separate licenses will allow you to analyze the needs of each community and tailor your interventions.
What is Client Coding?
Client Coding allows you to define six variables of your choice on which you can then sort clients for reports. Common variables are case manager, office location, education level, funder, family status, etc.
How do I evaluate interventions?
There is a Local Programs feature where you can enter the interventions you want to evaluate. Once you code a client to that Local Program and give the ERS to the client before and after the intervention, the ERS will generate a report on how much clients improved in that intervention. You can generate these reports as long as 10 or more clients are included.
If a staff member takes the ERS with a code set up on the Agency site in error, is it possible to remove their data?
No, it can't be removed. That's why we give two complimentary codes (“demxxxx”) with each license for staff use.
Are the access codes case sensitive?
Case sensitivity depends on how your browser is configured. We have not made them case sensitive.
How are clients linked to our license?
All Client User Names must start with your Client User Code, which links the client’s record to your Agency license for the Agency reports. The Client User Code is shown on the screen where you set up a new client.
If my client moves, can I transfer their ERS scores to another agency?
Yes, there is a Client Transfer feature. You can transfer a client as long as the other agency is licensed to use the ERS and accepts the client from you.
Can the Client Feedback Report be sent to another agency as a file attachment?
Yes, the Feedback Report is available as a pdf file, which can be saved and shared.
Can we control which takes are being compared?
The default for the report printed for the client is always #1 compared with the most recent take, but you can select a different comparison (#2 with #3, for example).
Can I say to a client, for example, “You are 90% there” or “You have progressed 10% over your last assessment”?
The Factor scores are “normalized” against the data from the field testing and there is a proportional "scale" embedded in the graph. You can compare different bars, but you cannot give numerical feedback.
Can the ERS feedback be delivered in a group setting?
Yes, you can give feedback in a group, and we have a Group Debrief protocol that you can use.
What other support do you provide?
Your Service Affiliate will monitor ERS usage to try to identify any implementation issues before they become problems.
What if we have our own training staff?
The ERS license states that we will provide staff training in order to ensure that it is accurate and that follow-up monitoring occurs. We are happy to include your training staff in the delivery of training, but will not transfer the responsibility for training to them.
What kind of training is available for managers?
Once you have sufficient client data, we provide training in running the aggregate client reports and in using the data for program planning and evaluation. We also provide PowerPoint presentations of your data to help you use it effectively.
What kind of training is available for staff?
We provide two stages of training. Part A is approximately 90 minutes and takes place when you are licensed, orienting you to how to use the ERS with clients. Part B training is approximately 60 minutes and takes place when at least 10 clients have taken the ERS; it helps you practice interpreting the Client Feedback reports. Additional training is provided on the various optional coding features of the ERS as desired. We will provide Parts A and B again if you have new staff, and we will also repeat Part B on request as a kind of case conferencing session to address staff questions that may arise in working with particular feedback reports. We also provide Training Notes as a guide for your staff. In addition to training, we provide telephone and email support to staff throughout the license period.
What client needs could we report to our funder if we are in the U.S.?
Of the U.S. clients who have taken the ERS, 59 percent were “Not Ready” (i.e., having less than a 40 percent chance of success without individual assistance) and only 16 percent were “Fully Ready” (i.e., only needing help with independent job search). The factors on which at least two-thirds of clients need assistance are (in order of importance) Job Search, Ongoing Career Management, Work History, Career Decision-Making, Skills Enhancement, Social Supports, and Self-Efficacy.
What client needs could we report to our funder if we are in Canada?
Of the Canadian clients who have taken the ERS, 65 percent were “Not Ready” (i.e., having less than a 40 percent chance of success without individual assistance) and only 13 percent were “Fully Ready” (i.e., only needing help with independent job search). The factors on which at least two-thirds of clients need assistance are (in order of importance) Ongoing Career Management, Job Search, Work History, Career Decision-Making, Skills Enhancement, Social Supports and Self-Efficacy.
How many agencies have used the ERS?
Since 2001, a total of 657 agencies have used the ERS, of which 531 were in Canada and 100 in the U.S.
How can we use the reports in reporting to our funder?
All of the Agency site reports of aggregated client data are intended to be printed and submitted to funders for any time frame you wish (e.g., monthly, quarterly, annually, year-to-year comparisons).
What support can we get in submitting funding proposals?
We would be pleased to provide wording (in addition to what appears on this site under “Information for Funding Proposals”) to help with funding proposals.
Is the Action Plan designed for the client or the agency?
The Action Plan is an optional part of the Client site that can be used in several ways. Agency staff can use it to develop a client action plan with the client and then to hold the client accountable by reviewing progress on the action plan at subsequent meetings. Or clients can be helped to start an action plan and expected to update it on their own.
What about having clients take the ERS on their own?
Because the ERS is an Internet-based tool, clients can be at any location that has Internet access. We have had licensees in rural or remote areas supply the client with their access code and have the client successfully take the ERS on a computer at home or at a local library or school.
Can the ERS be used to evaluate a ½ day or 1 day workshop?
Yes, as long as the workshop is focused on improving employability.
Is the ERS valid with psychiatric patients?
Yes, as long as the person can assess themselves reasonably realistically. If persons are severely paranoid, severely depressed, or severely manic, they need to be on appropriate medication before taking the ERS.
What is the impact of having staff read the questions to the client?
Based on field testing with persons not able to read either English or French or with reading comprehension problems, the only impact was lengthening the administration time from 15-20 minutes to 40 minutes. It helps if the client themselves click the mouse.
What about using the ERS with persons with disabilities?
Seven agencies working with persons with disabilities were part of the ERS field testing process. Staff reported that the only group for whom the ERS was not appropriate were persons with developmental disabilities as the concepts being tested were too complex. Persons who are very manic or very depressed or very paranoid need to be on appropriate medication (so that they self-assess realistically) before taking the ERS. Persons with visual or certain physical disabilities can use the ERS as long as they have the relevant adaptive technology (and enough computer memory to run both the adaptive technology and the ERS simultaneously). Staff reported no particular issues for persons with learning or mobility disabilities or those who were deaf or hard of hearing. Since the launch of the ERS, staff have reported that using the ERS helps them differentiate between disability-related issues and employment readiness issues.
What about using the ERS with high school students?
Using the ERS with juniors and seniors would be excellent (a) in order to determine what help they need in order to become employment ready, and (b) to evaluate career guidance programs (by having the students take the ERS before and after the program).
Is the scale valid with recent secondary school graduates?
Yes, the ERS is valid with persons between the ages of 14 and 65, based on the field testing.
What is the reading level of the scale?
7th grade comprehension.