Introduction to selecting validation methods

This section of the Catalogue of Validation Methods for learning outcomes is intended to answer the question: “How should the methods be selected for the identification, documentation and assessment of knowledge, skills and social competences?”

This question can be raised both when preparing the description of a qualification (and thus defining the conditions to be met by all entities awarding the qualification) and when a specific awarding body is designing a validation scenario.

The qualifications referred to herein are generally awarded to adults, who often want to continue learning, officially confirm their competences, retrain or return to the labour market.

Validation makes it possible to achieve these goals by placing a candidate in a situation where he/she can demonstrate what he/she knows and is able to do. The type of situation in which a person can prove their knowledge, skills and social competences depends on the method.

This section will show the process of selecting these methods and the factors underlying the choice.

The main elements to be considered when preparing a description of a qualification and a validation scenario

When preparing the description of a qualification

Guidance on the elements to be taken into account in the description of a qualification when determining validation requirements can be found in the publication The Validation of Learning Outcomes in Poland – New Opportunities for Attaining Qualifications [PDF]. The recommended procedure of selecting methods when preparing the description of a qualification is presented in the section entitled “Selecting methods when preparing the description of a qualification” (found in the tab The process of selecting validation methods).

It is worth remembering that different methods can be proposed at different stages of validation (identification, documentation and assessment of learning outcomes).

When preparing a validation scenario

When preparing a specific validation scenario, it is important to bear in mind that:

• The choice of methods depends on the learning outcomes and their assessment criteria as well as the conditions specified in the validation requirements included in the qualification’s description.
  
  The validation requirements contained in a qualification’s description may include, among others, information on the methods to be used at a given validation stage or those that should not be used. Presenting these requirements in a highly general form is recommended. This will enable the awarding bodies to develop their own solutions while ensuring that the validation results of different entities awarding the qualification are reliable and comparable.
  
  Experience to date shows that institutions describing qualifications often indicate specific validation methods in a qualification’s description. On the one hand, this limits the choice an awarding body can make when preparing a validation scenario. On the other hand, however, it may facilitate the accuracy and reliability of validation. In addition, it makes it possible to determine whether the awarding body has the appropriate resources to apply for the authorisation to award a qualification.
  
  
• More than one method may be used in the validation process – both at different stages and within the same stage (identification, documentation, assessment).
  
  The use of a combination of different methods in validation has been gaining in popularity in Europe for several years. The following variants are noted:
  
  
  • several methods are used in the same validation process, complementing each other – e.g. the analysis of evidence and statements presented in a portfolio is combined with an interview (unstructured or structured) or a debate (unstructured or structured) – this is the case in Denmark, France, Switzerland, Norway and Belgium;
  • depending on the sector in which validation occurs, different methods are used to validate the learning outcomes of the same qualification, such as in Finland;
  • the persons seeking validation or assessors can choose the appropriate method in a given situation from a pool of methods allowed for a given qualification – this is the case, among others, in the United Kingdom;
  • depending on the results of the first method, several different methods can be chosen at a later stage – this is the case in Germany and Finland, among others.
  
  Combining different methods is advisable because one method can rarely identify, document and assess all the learning outcomes. Similarly, knowledge, skills and social competences can rarely be confirmed using only one method. The use of several methods is particularly advisable for qualifications with a greater number of sets of learning outcomes.
  
  The use of differentiated methods ensures that the process is more reliable and accurate and makes it easier to adapt the process to the needs of the person seeking validation.
  
  
• Decisions concerning the choice of methods significantly impact the quality of validation.
  
  The recommended course of the process of selecting methods when preparing a validation scenario is described in the section “Selecting methods when preparing a validation scenario (tab The process of selecting validation methods).

The quality assurance criteria for validation

Box 1.The quality assurance criteria for validation

The choice of appropriate methods influences the outcome of the validation process and, indirectly, the quality of the awarded qualification. If the methods are wrong given the types of learning outcomes being confirmed, the results will not be credible. The use of the wrong methods can lead to the following situations:

• a certificate will be awarded to a person who has not achieved the learning outcomes required for the qualification;
• a certificate will not be awarded to a person who has achieved the learning outcomes required for the qualification (more on this subject can be found in the publication The Quality Assurance of Qualifications in the Integrated Qualifications System [PDF]).

Accuracy

What can be done during the method selection process to increase the likelihood that validation will be accurate? First of all, one should ensure that the method to be used is appropriate for the selected learning outcomes (regardless of whether this applies to all learning outcomes for a given qualification or a selected set). Among other things, one can:

• compare the possibilities and limitations of different methods in a given validation stage (identification, documentation or assessment) and in relation to the PQF levels of the learning outcomes;
• determine whether the learning outcomes can be identified, documented or assessed by other methods.

Care should also be taken to ensure that the methods used in successive validation stages complement each other. If one plans to use the analysis of evidence and statements, in the assessment stage, then the portfolio may be worth considering in the documentation stage, while methods that can assess the contents of that portfolio (e.g. observation in simulated conditions or in real-life conditions, a theoretical test).

Reliability

Reliability, understood as the comparability of the results of validation performed in different contexts and at different times, primarily depends on:

• using accurate and adequate methods in a given situation;
• selecting appropriate tools for a specific method – tools that are not properly developed or used can contribute to results that may not be reproducible and reliable;
• having a properly trained staff – staff must not only master validation methods, but also adhere to ethical conduct (e.g. equal treatment of candidates, compliance with the rules set forth in the validation scenario, avoid misleading the candidates, maintain confidentiality regarding assessment tasks and the candidate's answers, etc.);
• providing the organisational conditions, equipment and supplies required for the validation.

Adequacy

What can be done during the method selection process to increase the likelihood that validation will be accurate? First of all, one should ensure that the method to be used is appropriate for the selected learning outcomes (regardless of whether this applies to all learning outcomes for a given qualification or a selected set). Among other things, one can:

The validation scenario should also take into consideration:

• the stage of identifying a candidate's knowledge, skills and social competences when the description of the qualification does not indicate otherwise,
• the way in which the learning outcomes were achieved, the learning outcomes (credit points) confirmed thus far, and any limitations (e.g. in terms of physical fitness) of the candidate.

Most of these factors depend on the appropriate formulation of the validation requirements in the description of the qualification and their interpretation by awarding bodies when the validation scenario is developed (more on this can be found in the publication The Quality Assurance of Qualifications in the Integrated Qualifications System [PDF]).

At the same time, it should be remembered that the accuracy, reliability and adequacy of validation also depend on whether the methods are used individually or in combination with others.

The factors determining the choice of validation methods

When selecting the methods to be used in validation, the first factors to be taken into consideration should be:

1. the nature of the learning outcomes,
2. the validation stage,
3. the needs of the person seeking validation.

The different factors and how they influence the choice of validation methods are presented below.

infoIdentification – identifying and analysing the knowledge, skills and social competences that have already been acquired by the candidate; it may be conducted either independently or with the support of a validation counsellor;

infoDocumentation – gathering evidence proving the achievement of selected learning outcomes (e.g. in the form of certificates, documents attesting to completed internships, work samples, videos, recommendations, job descriptions, etc.); it may be conducted either independently or with the support of a validation counsellor;

infoAssessment – confirming the achievement of certain learning outcomes using the appropriate methods. In the case of validation aimed at having a specific qualification awarded, the results of the assessment are compared to the learning outcomes required for that qualification.

Download PDF
(Black and white)

The nature of the learning outcomes

Learning outcomes are knowledge, skills and social competences acquired through learning. The methods – and thus the techniques and tools used in validation – should correspond to the learning outcomes contained in the qualification and to the assessment criteria defined for them. This is a basic condition for a properly conducted validation, as well as for maintaining the accuracy of this process.

Each method used in validation is best suited to a specific category of learning outcomes.

In the assessment stage, the Polish Qualifications Framework level of the specific learning outcomes should also be considered. Some methods will work better at lower PQF levels, some will work better at the higher ones.

Validation stage

Diagram 1. Validation stages

The following stages are distinguished in the validation process:

• identification – identifying and analysing the knowledge, skills and social competences that have already been acquired by a candidate; it may be conducted either independently or with the support of a validation counsellor;
• documentation – gathering evidence proving the achievement of selected learning outcomes (e.g. in the form of certificates, documents attesting to completed internships, work samples, videos, recommendations, job descriptions, etc.); it may be conducted either independently or with the support of a validation counsellor;
• assessment – confirming the achievement of certain learning outcomes by using the appropriate methods. In the case of validation aimed at having a specific qualification awarded, the results of the assessment are compared to the learning outcomes required for that qualification.

Some methods can be used at each stage of validation (e.g. unstructured interview). Others are only relevant for specific stages (e.g. the skills audit in the identification and documentation stages). Even though some methods can be used in all stages, they are often best suited to one of them (e.g. theoretical test in the assessment stage).

Needs of the learner

In the context of validation, a learner's needs can cover many issues, such as facilitating access to the process (including, for example, in relation to people returning to the labour market) and providing comprehensive information about validation. However, when selecting methods, the following should be taken into account:

1. the candidate's learning path (achievement of learning outcomes),
2. the purpose for seeking validation.

Taking the above factors into account helps in designing validation appropriately, e.g. shortening the time of conducting validation and reducing the risk of failure for the candidate due to reasons other than the lack of required learning outcomes. On the other hand, more people can have access to validation.

Information on learners’ needs can already start to be collected while preparing the description of a qualification. When indicating the target groups for a qualification, it is worth considering typical candidates (e.g. will they be persons who have recently completed formal education, or rather those with some experience in the labour market). On this basis, decisions can be made on including requirements in the description about the identification and documentation of learning outcomes. Information on learners’ needs can also be helpful in selecting potential methods.

When preparing a validation scenario, it is worth including activities that identify a candidate's needs, e.g. during an initial interview with a validation counsellor.

Learning paths

Learning outcomes are basically achieved in three ways:

1. through formal education (school, university),
2. through non-formal education (training courses, workshops),
3. through informal learning (e.g. self-study, acquiring knowledge and skills at work and through hobbies).

Each of these paths may require a different approach in the validation process. This should be taken into account when developing the validation requirements in the description of the qualification, e.g. by considering the possible use of a variety of methods.

On the other hand, a validation scenario that respects different learning paths should include several options (alternative approaches) for candidates. For those who have achieved learning outcomes mainly outside formal education, the analysis of evidence and statements (portfolio)may be indicated, as it allows candidates to present their existing achievements. A theoretical test or observation in simulated conditions may be indicated for less experienced candidates. It is also possible to further differentiate methods within a specific validation scenario.

The learning path of a candidate may also require that validation take into account the stages of identifying and documenting learning outcomes. This is particularly important for people who have acquired a significant part of their knowledge, skills and social competences through practical experience. This may be due to the fact that:

1. at work, we often learn unintentionally, focusing primarily on completing specific assignments;
2. knowledge, skills and social competences used in work or for hobbies will not always be able to be straightforwardly transposed to the learning outcomes of a qualification, and if they are, this is not always evident or considered important;
3. sometimes the range of activities that are performed at work is not always equivalent to what is taught in the school or vocational training system (e.g. involving the use of the latest technologies); this makes it difficult to independently link the knowledge, skills and social competences used at work to existing qualifications, and thus may require the assistance of a validation counsellor.

It should also be remembered that the use of a specific method at a given validation stage can produce different results depending on how the candidate has achieved his/her learning outcomes.
There is a risk that persons who have acquired learning outcomes outside formal education over a long period of time will be stressed and resist the idea of taking a theoretical test. This may affect the result of the assessment. Fear of failure and judgement may “distort” the results or even cause the candidate to withdraw from the validation process before attaining the qualification. In such a situation, it is better to use observation in real-life conditions (e.g. workplace), the unstructured interview or analysis of evidence and statements.

On the other hand, persons who completed formal education fairly recently may not have difficulty in taking a theoretical test because of their experience and ease in doing so. However, due to their lower level of professional experience, using observation or the analysis of evidence and statements in their case could lead to poorer validation results.

The purpose of undertaking validation

The objective of validation in the Integrated Qualification System is to attain qualifications. However, each stage of the process produces results that may lead to measures serving additional aims, such as further learning or retraining (see Diagram 2).

Diagram 2. Results of subsequent validation stages and further steps

This means that someone who would like to identify their competence gaps (or determine their chances of attaining a qualification) can only participate in the identification stage and perhaps the stage of documenting learning outcomes. A person may also be able to determine how much has been learned (to note an increase in achieved learning outcomes). Therefore, the methods used at these stages should provide a candidate with comprehensive feedback on his/her current state of knowledge, skills and social competences. It is also worth remembering that the awarding body can also use these methods in the counselling process.

Just the opposite could also be the case – a candidate may only want to proceed with assessment, omitting the stages of identifying and documenting learning outcomes (if they are foreseen). When designing both the description of a qualification and a validation scenario, it is useful to take into account methods that are not dependent on other methods used in earlier stages (e.g. by preparing several validation approaches, one of which takes into account the results of having recognised a candidate's learning outcomes).

Factors limiting the choice of validation methods

Once the allowable methods have been determined, one should consider the relevant factors that may exclude some of them from a given validation stage.

When preparing the description of a qualification

Presented below are the factors that should be taken into account when developing the validation requirements in the description of a qualification takes place. However, as they are rather general, they should be reconsidered at the time of producing the validation scenario, and at that time, they should take into account the conditions prevailing in the awarding body.

These factors include:

1. the human resources required of the awarding body, including competence requirements for the validation process that will be implemented;
2. the required organisational and material resources;
3. the outlays needed for validation – this factor can only be defined in detail when the validation scenario is being prepared by a specific institution. However, while working on the description of a qualification, information on the required human, organisational and material resources in the sections on specific validation methods can be used to make an initial estimate.

When preparing the validation scenario

All awarding bodies must meet the personnel, organisational and material requirements set forth in the description of the qualification. However, when developing the validation scenario, its appropriate scheduling should be taken into account. This makes it possible to plan the validation properly and increase its effectiveness. Long waiting times for validation or excessive time extensions should be avoided (this can happen, for example, when only one employee/team is delegated to use a particular method).

Validation outlays include, among others, the costs of using a specific method in a given environment. This not only refers to the estimated cost of a candidate obtaining a certificate, but also the cost of preparing and using tools for specific methods. Some methods (e.g. observation in simulated conditions) can be very expensive to use, while others (e.g. unstructured interview) can be relatively low-cost.

The cost of preparing and continuously improving tools should also be taken into account – e.g. developing a suitably standardised theoretical test with an extensive database of questions is labour-intensive and costly. On the other hand, if interest in validation is likely to be high (e.g. in the case of qualifications awarded in industrial sectors), methods can be chosen that allow more candidates to be tested simultaneously. In such a situation, one must balance the costs and benefits of using different methods.

In addition, the following factors should be taken into account:

1. the expected number of candidates who may want to seek validation in a given period of time – this relates to appropriately scheduling the allocation of human, organisational and material resources;
2. the requirements of providing specific safety measures for the persons taking part in validation;
   
   While occupational health and safety regulations must be observed in all situations, they do not always influence the choice of methods. However, in a hazardous environment or in situations relating to the protection of health and life, assessing the learning outcomes should be performed in such a way as to minimise the risk of accidents. At such times, the use of observation in simulated conditions (simulation) is recommended, rather than observation in real-life conditions.
   
   
3. the place where validation is performed.
   
   For example, observation in real-life conditions requires an arrangement with an employer and will not always be possible (e.g. because of the hazardous working environment or the need to maintain professional discretion). On the other hand, observation in simulated conditions (simulation) may require the use of specific, specialised equipment. If only one awarding body has such resources, it will significantly reduce the ability to use this method. It should also be remembered that an awarding body is able to delegate the performance of validation to another entity, which may determine the choice of methods if this is allowed by the requirements for validation in the qualification’s description.

The factors limiting the use of methods will influence the decisions of the persons developing the validation scenario. However, these factors will not affect all qualifications. Moreover, in some cases, the use of a method may be determined by additional considerations, such as the desire to stand out from the competition, to reach specific target groups, or to fulfil the social mission of a given awarding body.

The process of selecting validation methods

The methods used in validation can be chosen at the time of:

1. preparing the description of the qualification – at the moment of defining the validation requirements,
2. developing the validation scenario in a specific awarding body, based on the requirements set forth in the qualification’s description.

The following is a sequence of actions to be taken in the process of selecting validation methods.

Selecting methods when preparing the description of a qualification

When preparing the description of a qualification, the purpose (demand, target groups), content (learning outcomes and assessment criteria) and manner of awarding the qualification (validation requirements) should be considered. More on this subject can be found in the paper How to describe market qualifications for the Polish Qualifications System. A Guidebook [PDF].

The entities describing qualifications should pay particular attention to validation because they are setting the standards for all awarding bodies when determining how a qualification can be attained.

The validation requirements in the description of a qualification should include only those elements required for a reliable validation result. Therefore, it may sometimes be useful to generalize the requirements by:

1. defining only general recommendations for validation, rather than specific methods;
2. indicating methods as recommended rather than required.

Required methods (as well as techniques and tools) directly translate into organisational, material and human resource requirements.

The following diagram presents a proposed sequence of actions to be taken in selecting methods when preparing the description of a qualification. It is based on the assumption that the qualification’s description should include requirements for specific methods (required or recommended).

Diagram 3. Activities to be undertaken in selecting appropriate validation methods when preparing the description of a qualification

1

2

3

4

5

Step 1 – Become familiar with the learning outcomes

When considering the choice of methods, the learning outcomes and their assessment criteria should be considered at each stage of validation.
It may also be helpful to analyse the operational verbs used. They name the activities that a candidate should perform to demonstrate that he/she has the required skills, knowledge and social competences.

When considering the learning outcomes in the context of the methods that can be used, the following questions should be answered:

1. Which type of methods can be used to conclude that an applicant for the qualification has achieved the learning outcomes [required for a qualification]?
2. Which type of methods will not allow us to determine whether an applicant for the qualification has achieved the learning outcomes [required for a qualification]?
3. Which methods enable the candidate to perform the activities indicated by the operational verbs included in the description of the qualification?

Return to the diagram

Step 2 – Decide which validation stages to include

Next, it is worth considering whether the description of the qualification should specify requirements for the identification and documentation stages, or only for the assessment stage.

The identification stage allows one to:

• identify the learning outcomes that have already been formally confirmed (meaning that they should not be re-confirmed);
• determine which learning outcomes can be confirmed;
• identify competence gaps.

As a result, the candidate – and potentially the awarding body – are better informed about an individual's chances of confirming the learning outcomes in a particular validation process. If competence gaps are identified, a further learning plan can be developed, which in turn may be one of the factors encouraging potential candidates to proceed with validation at the awarding body.

The documentation stage allows evidence to be collected and processed, demonstrating that a candidate has achieved the required learning outcomes in a way that allows them to be presented to a third party, e.g. for assessment. This is particularly useful if the assessment is based on the analysis of evidence and statements.

When describing a qualification, it is possible:

• to leave the requirements for the identification and documentation stages undefined, allowing this to be determined by the awarding body,
• to define the requirements for these stages,
• to determine which methods should not be used in these stages.

Answering the following questions can help in making a decision:

1. Given the target group or the nature of the qualification, should specific methods be proposed to identify or document a candidate's learning outcomes?
2. Given the target group or the nature of the qualification, should specific methods be used to identify or document a candidate's learning outcomes?

Return to the diagram

Step 3 – Determine the requirements for the methods used in the assessment stage

The choice of methods used in the assessment stage is primarily influenced by the learning outcomes and their assessment criteria (see Step 2).

Box 2. Elements helpful in identifying assessment methods

It is worth remembering that several methods can be used simultaneously in the assessment stage. This is particularly appropriate when the learning outcomes of a qualification are grouped into different sets (e.g. relating to planning, implementing activities, or evaluating results). In addition, many methods can be used together to complement each other, e.g. the analysis of evidence and statements together with the unstructured interview.

In formulating validation requirements, the persons describing the qualification sometimes may try to limit the choice to one basic method, even if the description indicates several methods. An example of this may be the sentence: “the assessment is conducted by using the interview with elements of a presentation, based on an observation or a piece of work made by the candidate".

The advantage of this formulation is its conciseness. However, there is a risk that it may be interpreted variably by different awarding bodies when the validation scenario is being developed. Although the description suggests the use of one method – unstructured interview (discussion), three additional methods are listed: presentation, observation in real-life conditions and analysis of evidence and statements (technique – assessing a piece of work). In addition, when reading this description, it may be difficult for the candidate to determine which requirements must be met before the assessment phase (preparation of the piece of work on which the interview is based) and which requirements must be met in the assessment phase.

The specific characteristics of a qualification’s target groups can also play an important role in selecting the methods. For example, the knowledge and skills of people with many years of professional experience can sometimes be assessed more effectively by analysing evidence and statements (e.g. the portfolio) than by tests. In the case of persons who have recently completed compulsory education, it may be the opposite.

When considering the selection of methods in the assessment stage, the following questions are worth answering:

1. Which type of methods can be used to determine that a candidate has achieved the learning outcomes required for the qualification?
2. Which type of methods will not allow us to determine whether a candidate has achieved the learning outcomes required for the qualification?
3. Which methods enable the candidate to perform the activities indicated by the operational verbs included in the description of the qualification?
4. Are the methods indicated for the assessment stage compatible with the methods indicated for the stages of identifying and documenting learning outcomes (if the description includes requirements for these stages)?
5. Does the selected method(s) allow the learning outcomes to be assessed regardless of how they were achieved?
6. Should a specific method (more than the other methods) be used because of the particular characteristics of the qualification’s target groups?
7. Should validation methods be specified for various sets of the qualification’s learning outcomes (e.g. the learning outcome sets have different qualities, some relate to theory and some to practice)?
8. Should one method be used for assessment or will using several methods together be more effective?

Return to the diagram

Step 4 – Determine the requirements for the method(s) used in the identification and documentation stages

Different methods can be used to identify and document learning outcomes. This should be remembered particularly when a break is planned between the stages or when they take place under different conditions (e.g. when validation is conducted in various places or by different people). At the same time, there are also methods suitable for both stages, e.g. the skills audit.

In addition, the persons preparing the description should ensure that the methods used in the different stages of validation are compatible with each other. For example, a portfolio allows evidence to be collected of a candidate’s achieved learning outcomes. If this method appears in the documentation stage, the assessment stage should include the analysis of evidence and statements to evaluate the content of the portfolio.

The individual needs of a candidate should also influence the choice of methods used in the identification and documentation stages. Some of these needs can already be foreseen when describing a qualification, provided that the specific characteristics of the qualification’s target groups are taken into account.

When considering the selection of methods in the identification and documentation stages, the following questions are worth answering:

1. Which type of methods can be used to identify the learning outcomes achieved through different learning paths by an individual?
2. Which type of methods can be used to document the achievement of learning outcomes?
3. Do the methods indicated for the identification and documentation stages allow evidence to be generated about achieved learning outcomes that can then be assessed?
4. Given the specific characteristics of the qualification’s target groups, should a specific method be used to identify or document learning outcomes (more than the other methods)?
5. Given the nature of the learning outcomes, should different methods be used in the identification and documentation stages, or will one and the same method enable knowledge, skills and social competences to be determined, and generate the evidence for their achievement?
6. Is the method(s) used in the identification and documentation stages compatible with the method(s) indicated for the assessment stage?

Return to the diagram

Step 5 – Confirm that the methods selected for the specific validation stages are correct

The last step in the process of selecting methods for the formulation of validation requirements in the qualification’s description is to make sure that everything required for the validation process is correctly chosen – the selected methods as well as the required resources.

The methods used at the different stages should contribute to the:

• accuracy,
• reliability, and
• adequacy of validation (see Box 1 in the Introduction to the selection of validation methods).

Answering the questions proposed in steps 3 and 4 again may be helpful at this point.

Finally, one should verify that the requirements for the method(s) are consistent with the organisational, material and personnel requirements (e.g. the description of the method does not imply the need to involve a person with specific competencies who has not been included in the validation scenario). The sections on each method include information on this issue.

Return to the diagram

Selecting methods when developing the validation scenario

The validation scenario should describe and facilitate the aim and course of the validation process of a specific qualification for all parties involved: the persons performing the validation, candidates and institutions.

The basic source of information about the requirements for validation methods is the description of the qualification (cf. the publication How to describe market qualifications for the Polish Qualifications System. A Guidebook [PDF]). Furthermore:

• in the process of describing and including a qualification in the IQS, the relevant minister may specify additional requirements for awarding the qualification,
• the awarding body may specify general recommendations and define its own requirements for awarding a qualification.

 Taking into account candidates' needs in the process of selecting the methods

Unlike the information in the description of a qualification, the validation scenario should provide information about the specific methods, techniques and tools to be used in the validation process. This means that more detail is required about the validation requirements contained in the qualification’s description.

This also means that the needs of the learner should be taken into account when developing the scenario, including:

1. the learning path,
2. the purpose of seeking validation (cf. tab Factors determining the choice of validation methods).

This information can be obtained, for example, by interviewing the candidate or through a detailed questionnaire on a website. Interviews may be conducted as part of the services provided to candidates.

Box 3. Services provided to a candidate

Proposed sequence of activities

The proposed sequence of activities for selecting methods when developing the validation scenario is presented in the diagram below.

Diagram 4. Activities to be taken in the selection of appropriate validation methods when developing the validation scenario

1

2

3

4

Step 1 – Become familiar with the qualification’s description

When determining the methods to be used in the validation process, one should become thoroughly familiar with the description of the qualification, including its learning outcomes, assessment criteria and validation requirements.

It may be helpful to analyse the operational verbs used in the description (mainly in the section on assessment criteria). They refer to the activities that a candidate should perform to demonstrate that he/she has achieved the required skills, knowledge and social competences.

The candidate's needs should also be identified.

While reviewing the learning outcomes in the context of the methods that can be used, it is recommended that the following questions be answered:

1. What are the candidate's needs with regard to validation? Do the candidate's needs and abilities influence the choice of methods (e.g. the purpose of seeking validation, the learning path of acquiring learning outcomes, work experience, achievements)?
2. Which learning outcomes should be assessed in this particular validation process – all or only some of them?
3. Are the requirements for the stages of identifying and documenting learning outcomes defined in the qualification’s description? If not, does the awarding body intend to conduct these stages?
4. How many people may be interested in seeking validation in the awarding body over a given period of time (e.g. month/quarter/year)? Is it possible to predict how many people will be interested only in assessment and how many in identification and documentation during this period? Answers to these questions influence the next ones:
   1. Given the expected interest in validation in a given period of time, does the awarding body have a sufficient number of people with the necessary competences to use the selected methods, or are additional measures needed (e.g. increasing employment, changing to another method)?
   2. Given the expected interest in validation in a given period of time, does the awarding body have the sufficient organisational and material resources needed to use the selected methods, or are additional measures needed?

Return to the diagram

Step 2 – Select the appropriate methods for the identification and documentation stages

When validation of a qualification’s learning outcomes includes the stages of identification and documentation, this should be taken into account when choosing the methods. It is essential that the methods are compatible in the identification, documentation and assessment stages if more than one method is to be used in the process.

In addition, when developing a validation scenario, it is worth considering whether:

• the identification and documentation stages are designed to prepare the candidate for the assessment stage (i.e. to determine whether his/her knowledge, skills and social competences correspond to the qualification’s requirements and to appropriately prepare evidence of this) or whether the candidate has needs exceeding the attainment of the qualification;
• the awarding body intends to conduct a diagnosis of broadly understood competences during these stages.

The scenario should also specify whether these stages can take place at different times and under different conditions (e.g. the candidate identifies his/her own learning outcomes using a dedicated web-based tool and the validation counsellor supports him/her in gathering relevant evidence). In such a situation, the methods used for identification may differ from those used for documentation.

When considering the selection of methods in the identification and documentation stages, it is worth answering the following questions:

1. Do the methods used in the identification and documentation stages differ from those used in the assessment stage?
2. Which methods are best suited to identify the learning outcomes achieved by an individual through different learning paths?
3. Which methods will best document the achievement of learning outcomes?
4. Should more than one method be used at each of these stages to ensure the accuracy, reliability and adequacy of validation?
5. Are the methods compatible with each other? If several methods are planned in different stages of validation, does the use of one of them in one stage preclude the use of another in subsequent stages? Should one method be used in an earlier stage in order to use a specific method later on? Are there methods that can only be used if another method(s) is (are) applied at an earlier stage?

Return to the diagram

Step 3 – Selecting the appropriate methods for the assessment stage

Step 3 concerns the assessment of learning outcomes. At this stage of validation, the choice of methods should be dictated primarily by the nature of the learning outcomes (and assessment criteria) and the methods recommended or specified in the qualification’s description.

When considering the selection of methods in the assessment stage, it is worth answering the following questions (this is especially true when the qualification’s description allows the awarding bodies to freely choose the assessment methods):

1. Do the selected methods correspond to the nature of the learning outcomes and allow the assessment criteria specified for the qualification to be confirmed?
2. Are the selected methods optimal in terms of the scope of the assessed learning outcomes and their Polish Qualifications Framework level?
3. Should more than one method be used at this stage to maintain the accuracy, reliability and adequacy of the methods used?
4. Do the selected methods take into account the candidate's needs?
5. Are the methods used at different stages of validation compatible with each other? If the validation process also includes the identification and documentation stages, do the methods selected for the assessment stage allow the results of the previous stages to be taken into account?
6. Does the use of the method enable specific safety measures for candidates and assessors to be observed (if required by the specific qualification)?

Return to the diagram

Step 4 – Confirm that the methods selected for the specific validation stages are correct

The last step in the process of selecting methods when developing the validation scenario is to make sure that the process has been performed correctly.

The methods used at the different stages should contribute to the:

• accuracy,
• reliability, and
• adequacy of the validation process (see Box 1 in the tab Introduction to the selection of validation methods).

Answering the supplementary questions proposed in steps 2 and 3 again may be helpful at this point.

One should also make sure that:

• the methods comply with the validation requirements contained in the qualification’s description;
• the available human resources as well as the organisational and material resources of the awarding body allow these methods to be used in the long term.

The sections describing each method will be helpful in determining this.

Return to the diagram

Bibliography

Dybaś, M. (ed.). (2015). The Quality Assurance of Qualifications in the Integrated Qualifications System [Zapewnianie jakości procesu nadawania kwalifikacji spoza systemów oświaty i szkolnictwa wyższego]. Warsaw: Educational Research Institute

European Commission; Cedefop; ICF International (2014). European inventory on validation of non-formal and informal learning 2014. Thematic report: validation methods. Luxembourg: Publications Office. Downloaded from: https://cumulus.cedefop.europa.eu/files/vetelib/2014/87240.pdf

Gmaj, I., Grzeszczak, J., Leyk, A., Pierwieniecka, R., Sławiński, S., Tauber, M. and Walicka, S. (2016). The Validation of Learning Outcomes in Poland – New Opportunities for Attaining Qualifications [Walidacja – nowe możliwości zdobywania kwalifikacji ]. Warsaw: Educational Research Institute

Sławiński, S. (2017). Słownik Zintegrowanego Systemu Kwalifikacji [Glossary of the Integrated Qualification System]. Warsaw: Educational Research Institute

Ziewiec-Skokowska, G., Danowska-Florczyk, E., Stęchły, W. (ed.). (2016). How to Describe Market Qualifications for the Polish Qualifications System. A Guidebook [Opisywanie kwalifikacji nadawanych poza systemami oświaty i szkolnictwa wyższego. Poradnik]. Warsaw: Educational Research Institute

Strony internetowe:

Cedefop (2016). European inventory on validation of non-formal and informal learning. Downloaded from: http://www.cedefop.europa.eu/en/publications-and-resources/country-reports/european-inventory-on-validation