Print

How operational verbs can help in choosing validation methods


What are operational verbs?

Operational verbs are those that describe actions which can be observed, confirmed and assessed. In other words, they are used to describe activities, which are to be measured.

Verbs of this type are used primarily to describe:

  • the learning outcomes of qualifications,
  • the corresponding assessment criteria.

This makes validation possible – confirming whether a person (regardless of the learning path) has achieved the required knowledge, skills and social competences.

Operational verbs are also used to describe the levels of the Polish Qualifications Framework (PQF). As a result, they influence, among other things, determining the level assigned to a given qualification.

These verbs are also used in formal education.

Box 1. The use of operational verbs in formal education

Approximately 150 operational verbs are used in formal education. They describe which learning outcomes a student should demonstrate at the end of a given educational stage. On this basis, test items are developed to determine the level of mastery of the material in the core curricula. They are also taken into account in developing tools and selecting examination techniques, as well as didactic measurements and in evaluation.

If the requirements of the core curriculum of a given subject are defined with the use of verbs such as “name”, “define”, “list”, then the knowledge and mastery of phenomena are assessed. In the case of the verbs “summarise”, “explain”, “distinguish”, the understanding of phenomena is confirmed, but not the ability to use knowledge in practice.

The verbs can point to typical (“construct”, “use”, “solve”) or problem-related situations (“plan”, “detect”, “anticipate”).

Operational verbs can be assigned to five categories indicating the type of skills being assessed:

  1. analytical (analyse, examine, diagnose, prove, evaluate, classify, interpret, identify, assess, forecast, detect, compare, calculate, generalize, deduce, justify, differentiate),
  2. verbal and communicational (characterise, inform, explain, criticise, describe, present, summarise, explain, demonstrate),
  3. organisational (plan, lead, draw up, maintain, deliver, collect, store, sort, prepare, provide services, set up, gather, prevent),
  4. technical and manual (measure, construct, design, adjust, obtain, make, select, use, link, insert),
  5. managerial (decide, direct, integrate, mobilise, control, develop, follow, set forth, designate).

A list of example operational verbs can be found in the publication How to describe market qualifications for the Polish Qualifications System. A Guidebook. They are often based on Bloom’s taxonomy.

The analysis of operational verbs facilitates, among other things, the selection of appropriate validation methods and tools, as well as the determination of a qualification’s PQF level.


Selecting validation methods depending on the operational verb used

The methods used in validation should correspond to the qualification’s learning outcomes and to their assessment criteria. This is a basic condition of a properly conducted validation, and also serves to maintain the accuracy of this process.

The operational verbs used to formulate learning outcomes and the assessment criteria describe the steps to be taken to demonstrate the achievement of a given learning outcome. Thus, they can suggest the choice of the validation method and in consequence – the right tool. This is one of the reasons why no more than one operational verb should be used in a learning outcome in the assessment criterion.

In addition, operational verbs can help determine the context in which the learning outcomes are assessed, especially the limitations of this process.

Example

The verb “clean” indicates that in the assessment process an object should be used that the candidate will be able to clean. The verb “cooperate” determines the presence of others; “isolate” may suggest the need to ensure occupational health and safety regulations.

Verbs in the verbal and communicational skills group (e.g. “explain”, “criticise”, “describe”) mean that candidates with special needs (e.g. persons with speech or hearing disabilities) should be provided with conditions under which it will be possible to assess the learning outcomes described with these verbs.

Note that some operational verbs have many meanings. The same verb may refer to knowledge, skills or social competences, depending on how the rest of the learning outcome/assessment criterion is formulated. Consequently, this affects the use of different assessment methods and tools. Therefore, when analysing the operational verbs in the context of validation methods, a description of the entire learning outcome should be taken into account.

Example

The verb “indicate” can refer to knowledge if the learning outcome specifies that the candidate should indicate e.g. which parts of a car engine are prone to wearing out. It can also refer to skills if the candidate has to indicate a worn out engine part and replace it with a new one.

In the first case, a theoretical test (written test or oral assessment) and an interview (unstructured or structured) may be the appropriate assessment methods. In the second case – observation in real-life or simulated conditions (practical assignment) and the analysis of evidence and statements (a piece of work or portfolio assessment).

The operational verbs used may indicate the form and method of solving an assessment tasktem used to assess a given category of learning outcomes.

Example

Test items or problems relating to knowledge can be asked in open, closed, single or multiple choice forms. In the case of skills, the way of performing an assigned task requiring the use of knowledge in typical or non-routine conditions is assessed. The verbs used to describe social competences indicate the situation in which a candidate should be placed in order to confirm his/her readiness to demonstrate a certain behaviour.

 

Using a given operational verb is not equivalent to using a specific method. However, it narrows the choice to the method(s) best suited to a particular learning outcome or assessment criterion.

In order to illustrate how the operational verbs used in the descriptions of qualifications can help in choosing validation methods, an example of selected entries from the draft version of a paper prepared by the French-Polish Chamber of Commerce is presented below. It indicates methods that can be used to assess the learning outcomes of the qualification entitled “Actively sell products and services to business clients” (October 2017 version).

Table 1. Methods proposed for the assessment of selected learning outcomes

Set of learning outcomes Selected learning outcomes Selected assessment criteria Proposed methods
Plan sales activities and monitor their results Draws up a comprehensive plan of sales activities based on the objectives received Discusses the indicated market segment based on the information provided, indicating the characteristics of this market from the point of view of sales Debate
Plans one’s sales activities based on the objectives and budget assumptions and other guidelines received Simulation
Prepares the meeting Identifies the decision-makers in the client's company Test
Presents ways of reaching out to decision-makers and establishing relationships with them Interview
 
Conduct the sales process Initiates contact with the client, taking into account the guidelines of the organisation Enumerates the principles of initiating contact with clients and illustrates them with examples Interview
Discusses the influence of a salesperson's image on building relationships with the client Debate
Establishes relationships with the client, using communication techniques Simulation
Formulates the scope of the purchase order based on the diagnosed situation of the client
Determines the terms and conditions of the offer within the given boundary conditions in the organisation Enumerates the principles of presenting the offer price Interview
Explains the importance of communicating the benefits of the various features of the offer Interview
Selects sample questions to diagnose the client's opinion about the offer according to prepared templates Test
Accepts the purchase order and completes the transaction Indicates the elements required to prepare a summary of the agreed items and obtain acceptance of the purchase order Test
Discusses an example of how the purchase order can be delivered to the organisation (including to whom and in what form) Discussion
 
Provide aftersales support Provide aftersales support Discusses the importance of maintaining a long-term relationship with the client, including formulating a request to the client to recommend him/her to other clients Discussion
Enumerates the tools allowing further contact with a potential client (including: CRM, mailings) Test
Identifies areas of potential additional sales to a given client Simulation
Monitor the completion of the purchase order Discusses his/her role in the process of purchase order completion, including the handling of complaints Discussion
Describes debt collection activities Test
Supports the client in using, promoting or selling the products or services provided on the basis of a scenario developed by the organisation Enumerates the means of using, promoting or selling the products or services provided Interview
Selects the means of assisting the client in using, promoting or selling the products or services provided

 

The assessment criteria described in the table above can be assessed by means of: observation in simulated conditions (simulation; a total of 8 criteria), discussion (3), debate (6), interview (6) and test (9). One to three of the methods enumerated above should be used to assess each of the learning outcomes.

The analysis of the assessment criteria (described using operational verbs) with regard to the selection of methods allows tools to be developed to confirm the learning outcomes.

 

Example (based on the example above)

The scenario of the interview at the assessment stage should cover three topics relating to the three assessment criteria:

  • discussion of an example of how the purchase order can be delivered to the organisation (including to whom and in what form),
    discussion of the importance of maintaining a long-term relationship with the client (including formulating a request to the client to recommend him/her to other clients);
    discussion of the candidate's role in the process of concluding the purchase order (including the handling of complaints).

This does not mean designing three separate interviews, as one interview can cover these three areas. The time required to assess each criterion needs to be estimated.

By estimating the time needed to assess a criterion, the approximate length of time it takes to assess one outcome or one set of learning outcomes can be determined.


The use of operational verbs in the Polish Qualification Framework

Operational verbs are used to formulate some of the descriptors of the Polish Qualifications Framework levels. These descriptors are general statements about the knowledge, skills and social competences required of qualifications at a given level. The descriptors of subsequent levels reflect the increasing requirements indicated in the learning outcomes.

However, not every verb in the framework is an operational verb, i.e. not every verb refers to actions that can be observed and measured. Examples of such verbs:

These are used to define the categories of learning outcomes in the PQF.

Apart from the fact that they do not describe measurable activities, both verbs describing the category of knowledge (“know”, “understand”) need to be supplemented.

Example

“Knows and understands methods and technologies relating to the occupational activities being performed” – PQF level 2 in terms of “methods and technologies” (second stage generic descriptors of the PQF typical for vocational education and training).

Analysis of the use of operational verbs

The following analysis is based on the characteristics of the second stage generic descriptors of the PQF typical for vocational education and training.

The requirements for specific PQF levels are categorised by knowledge, skills and social competences. In order to specify them, descriptive categories have been added to each category of learning outcomes and basic aspects further specify each descriptive category.

category of learning outcomes

(e.g. social competences)

descriptive category

(e.g. cooperation)

basic aspect

(e.g. communication)

basic aspect

(e.g. professional relations)

descriptive category

basic aspect

basic aspect

 

As the requirements for the different levels of the PQF change, different operational verbs are used. This can be traced in the example below.

Example

The following operational verbs have been used to show how the requirements concerning the ability to use work materials and tools (“using and selecting” aspect) increase at successive PQF levels:

“use” (PQF levels 1 and 2), 
“organise” (PQF level 3), 
“design” (PQF level 4), 
“select” (PQF level 5), 
“adapt” (PQF level 6), 
“modify” (PQF level 7), 
“develop” (PQF level 8),

The increasing requirements at successive PQF levels are determined not only by operational verbs, but also by their objects, which indicate the degree of difficulty of the activity performed. In this example there are as many as 8 nouns of various kinds: “tools”, “equipment” (PQF level 1), “workstation equipment” (PQF level 2), “workstation” (PQF level 3 and 4), “materials” (PQF level 5), “procedures” (PQF level 5, 6 and 7), “methods”, “technologies” (PQF levels 5, 6, 7 and 8).

 

On this basis, it is possible to trace how a skill changes depending on the PQF level: from “is able to use the tools and machines required to carry out elements of very simple occupational activities in compliance with safety requirements” (PQF level 1) to “is able to develop new methods and technologies for the occupation” (PQF level 8).

The variability of a given skill (knowledge and social competences) described in the PQF level descriptors makes it easier to determine the level of a specific qualification.

Example

If the qualification contains learning outcomes and/or assessment criteria relating to “designing a work station for carrying out moderately complex occupational tasks” (requirements for PQF level 4), then they correspond to PQF level 4.

In such a situation, a person with this skill may also be expected to use typical tools and equipment (requirements for PQF level 1 and 2) and organise a work station (PQF level 3).

However, it should be remembered that sometimes the same operational verbs have been used in the descriptors of different PQF levels or in relation to different basic aspects.

Example

The verb “develop” was used at PQF level 5: “a person is able to develop and keep records relating to the documentation required for the occupation” (descriptive category of “information”, “documentation” aspect).

The same verb was used to describe the requirements for PQF level 7 relating to the ability to “develop a strategic plan for a work team/organisation relating to the work of the occupation” (the “planning and revising plans” aspect of the “organising work” descriptive category).

Therefore, the PQF level should not be based solely on operational verbs. The full description of the learning outcomes and the requirements of a given PQF level must be taken into account.

The use of operational verbs in the PQF level descriptors makes it possible to determine which validation methods can be used to confirm the learning outcomes meeting the requirements of a given PQF level.

 

Example

Several methods can be recommended to confirm the skills relating to the “using and selecting” at each level of the PQF.

Table 2. Methods of validation recommended in relation to PQF level descriptors

PQF level PQF level descriptor
learning outcomes category: “skills”
aspect: “using and selecting”
Recommended validation methods
1 [The candidate is able to] use the tools and machines required to carry out elements of very simple occupational activities in compliance with safety requirements.
  • observation in simulated conditions (simulation)
  • observation in real-life conditions
  • analysis of evidence and statements
2 [The candidate is able to] use the routine equipment of a work station for carrying out simple occupational activities in compliance with safety requirements.
  • simulation
  • observation in real-life conditions
  • analysis of evidence and statements
3 [The candidate is able to] organise a work station for carrying out not very complex occupational tasks in compliance with safety requirements.
  • simulation
  • observation in real-life conditions
  • analysis of evidence and statements
4 [The candidate is able to] design a work station for carrying out moderately complex occupational tasks.
  • simulation
  • observation in real-life conditions
  • analysis of evidence and statements
  • theoretical test
5 [The candidate is able to] select the methods, technologies, procedures and materials required for the performed occupation.
  • simulation
  • observation in real-life conditions
  • analysis of evidence and statements
  • theoretical test
  • unstructured interview
  • structured interview

auxiliary methods (used in combination with others):

  • unstructured debate
  • structured debate
6 [The candidate is able to] adapt simple methods and technologies as well as simple procedures in the performed occupation.
  • simulation
  • observation in real-life conditions
  • analysis of evidence and statements
  • theoretical test

auxiliary methods (used in combination with others):

  • unstructured debate 
  • structured debate
7 [The candidate is able to] modify methods and technologies as well as procedures for the occupation.
  • simulation
  • observation in real-life conditions
  • analysis of evidence and statements
  • unstructured debate
8 [The candidate is able to] develop new methods and technologies for the occupation.
  • simulation
  • observation in real-life conditions
  • analysis of evidence and statements
  • unstructured debate
  • presentation

 

Learning outcomes corresponding to “[the candidate is able to] develop and keep records relating to the documentation required for the occupation” (PQF level 4 requirements) could be confirmed by means of: observation in real-life or simulated conditions (practical assignment) and the analysis of evidence and statements. The presentation could also be used, but the conclusions drawn from it should be supported by other methods, more focused on testing practical skills.

On the other hand, the requirements concerning the ability to “develop a strategic plan for a work team/organisation relating to the work of the occupation” (PQF level 7) assume the ability to plan and revise plans within a work organisation (team or company). They include, among others: planning professional activities, revising them according to circumstances, taking into account changing business conditions, having an understanding of one’s employees (their skills, directions of professional development), analysing information and data and forecasting on their basis.

The methods recommended for confirming the learning outcomes corresponding to such a complex PQF level descriptor include: observation in the workplace and observation in simulated conditions (e.g. simulation games, project, practical assignment). To complement these, the structured debate or presentation can also be used (in order to identify learning outcomes that are revealed in group processes or in individual work).


Guidelines