SAQA: Quality Management Systems for ETQAs

Download a copy of this policy form the SAQA website here:

http://saqa.org.za/docs/pol/2003/qmsetqa.pdf

Quality Management Systems for ETQAs

P0LICY DOCUMENT

Please refer any queries in writing to:

The Executive Officer

SAQA

Director: Quality Assurance and Development RE: Quality Management Systems for ETQAs Postnet Suite 248

Private Bag X06

Brooklyn 0145

Pretoria

South Africa Fax (012) 346 5809 e-mail: saqainfo@saqa.org.za

Publication date: October 2001

ISBN: 0958441952

Funded by the European Union under the European

Programme for Reconstruction and Development

Quality Management

Systems  for ETQAs

THE SOUTH AFRICAN QUALIFICATIONS AUTHORITY

Table of Contents

Executive Summary                                                                            5

1  Introduction                                                                                   7

2  Quality Terminology                                                                       9

3  Quality Management Systems                                                    12

4  A Quality Management System for the National

Qualifications and Standards Framework (NQF)                      15

5  The ETQAs Role in the Quality Management

System for the NQF                                                                     20

6  Variations in the Contexts in which the ETQAs serving

different sectors will operate that may impact on their QMS   36

Appendix 1: The Purposes of Learning                                         46

Definition of Terms                                                                          47

Bibliography                                                                                     48

Notes, Tables and Diagrams

Note 1:       A Note about Cost                                                         11

Table 1:       Characteristics of a Quality Management System            13

Diagram 1: Quality Spirals                                                               14

Table 2:       Managing Quality in the NQF System                              16

Table 3:       The Quality Management Roles of ETQAs within

the NQF System                                                            21

Table 4:       Activities associated with the Quality Management

Roles of ETQAs                                                            23

Table 5:       Some differences between ETQAs from the two

identified sectors that will influence their QMS.               44

Table 6:       Primary Reasons for Learning                                         46

Executive summary

E

ducation and Training Quality Assurance Bodies (ETQAs), while independent bodies, by definition exist as an integral part of the system established to introduce and implement the National Qualifications Framework (NQF). Along with the other components of the system (SAQA, NSBs, SGBs, Moderating bodies and Providers), ETQAs are a part of the NQF quality

management system.

ETQAs in the two identified sectors, (Education and Training sub-system and Economic sectors), have to deal with providers of different sizes and cultures, who frequently will provide learning for different purposes in terms of different qualifications and standards. The ETQAs in these two sectors are likely to vary in size and in their wider roles beyond the NQF. While the fundamentals of a quality management system for ETQAs are relevant to all, there will be variations between the sectors. It is unlikely that one model of quality assurance management will suit all. The outcomes, however, should be the same; the development of a quality culture that benefits learners and society as described by the objectives of the NQF.

Quality management depends on creating a quality culture, which amongst other things, means everyone accepts full responsibility for quality and has the flexibility to respond to their particular situation.

For these reasons, SAQA is not going to prescribe a detailed quality management system for

ETQAs.

However, any ETQA quality management system will include the following essential roles:

  • Create and sustain a quality culture.
  • Contribute to ensuring the relevance, comprehensiveness and clarity of qualifications and standards.
  • Confirm that providers ensure that the facilitators of learning and/or assessment have the requisite skills.
  • Confirm that providers regularly monitor and report on the quality and effectiveness of learning and qualifications and standards.
  • Confirm that the providers ensure that practices are enhanced in the light of what is learned from monitoring activities.
  • Confirm that suitable resources are available and are used to good effect.
  • Regularly seek, receive and act on feedback from their customers: SAQA, providers, SGBs, NSBs and stakeholders.
  • Monitor providers’ outcomes and internal audit process and report back to providers, SAQA and NSBs.

There will be some questions and considerations common to each of these that all ETQAs will need to address. These are identified and discussed, as are some of the key differences in the contexts in which the ETQAs serving the education and training sub-system and economic sector-learning environment will operate.

Quality Management Systems for ETQAs                                                                                                            5

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

ETQAs, together with SAQA, play a pivotal role within the quality management system of the NQF system. They can foster a quality culture within the NQF System both through creating such a culture within their own organisation and through helping and encouraging providers to do the same. However, if they are to foster a quality culture, they will depend on the other parts of the NQF system (SAQA, NSB/SGB, Providers and Moderating Bodies) to play their part and will need to help them to do so.

In order to foster a quality culture amongst providers ETQAs are likely to use a combination of:

  • Initial accreditation that requires providers to undertake a range of assessments (e.g. self and peer) and quality assurance activities and to report on the outcomes of these activities.
  • Monitoring activities that include the direct auditing of reports, systems, processes and outcomes and the judicious use of technically sound external evaluation.
  • The use of technically sound assessments to confirm the knowledge and skills of learners, especially where the associated qualifications and standards are to inform high-stakes decisions.
  • Annual reviews that include establishing and agreeing with providers’ new quality goals, taking into account the previous year’s achievements and the outcomes of monitoring

activities.

6                                                                                                           Quality Management Systems for ETQAs

Chapter 1

Introduction

R

egulation R1127, under the South African Qualifications Authority Act, 1995 specifies the roles and responsibilities  of Education and Training Quality Assurance Bodies (ETQAs). ETQAs are accredited by the South African Qualifications Authority (SAQA)

for:

“The purpose of monitoring and auditing achievements in terms of national standards or qualifications and standards.”

The ETQAs, while independent bodies, by definition exist as an integral part of the system established to introduce and implement the National Qualifications Framework (NQF). Along with  the  other  components  of  the  system,  SAQA,  National  Standards  Bodies  (NSBs), Standards Generating Bodies (SGBs), Moderating Bodies and Providers, ETQAs are a part of the NQF quality management system. However, as independent organisations, each ETQA is responsible for the quality of the services it provides to its customers and therefore must maintain its own internal quality management system.

The ETQA Regulation (R1127) gives the following definition of a quality management

system:

“The combination of processes used to ensure that the degree of excellence specified is achieved.”

To paraphrase the objectives specified by the Act, the ultimate purposes of the system are to:

  • Enhance learning in South Africa by increasing the number of learners, the frequency of learning, the amount they learn, and the relevance and durability of what is learned.
  • Establish a framework of qualifications and standards that are relevant, credible and accessible.

Relevant learning is defined as that which contributes to:

  • Developing the full personal potential of each learner and the social and economic development of the nation at large.
  • Facilitating mobility and progression within education, training and career paths. The credibility of a qualification depends on:
  • Its relevance (or fitness for purpose).
  • The confidence of those who use it as an indicator of the skills and knowledge of the holder.
  • Its accessibility to those who aspire to gain it.

With regard to accessibility generally, the Act specifically requires the system to accelerate the redress of past unfair discrimination in education, training and employment opportunities. This document is a detailed examination of quality management systems.

Quality Management Systems for ETQAs                                                                                                            7

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

The following section explains quality-related terminology. The subsequent sections suggest an  approach  to  quality  management  for  the  NQF  system  as  a  whole  followed  by  an examination  of  the  ETQAs’  role  within  the  system.  Lastly,  possible  differences  in  the challenges that face ETQAs in different sectors, and the implications of how they fulfill their role, are discussed.

8                                                                                                           Quality Management Systems for ETQAs

Chapter 2

Quality Terminology

“Q

uality” has become a discipline in its own right, and, like many disciplines, terms assume very specific meanings. In many contexts, where experts will understand the differences  between  the  terms;  many  lay  people  will  think  they  are  synonymous.  The following  explanations  of key terminology  are offered to help clarify understanding  and

stimulate dialogue.

Quality Management Systems (QMS)

The ETQA Regulation (R1127) gives the following definition:

“The combination of processes used to ensure that the degree of excellence specified is achieved.”

A more general explanation of the purpose of a QMS would be as follows:

“A quality management system is the sum of the activities and information an organisation uses to enable it to better and more consistently  deliver products  and services that meet and exceed the needs and expectations of its customers and beneficiaries, more cost effectively and cost efficiently, today and in the future.”

Ultimately QMS is about creating a “quality” culture across an organisation.

Key considerations in QMS are securing continual improvement in quality, today and in the future. It is about more than sustaining quality or even assuring quality today. It is about maximising the ability of the organisation to consistently deliver high quality products and services into the future, in changing circumstances. Quality Assurance, Quality Audit and  Quality Control are elements  of, but not the totality  of, a  Quality Management System.

Critical elements of a quality management system that are not normally associated with quality assurance/audit/control are those concerned with:

  • Enhancing quality, cost effectiveness and efficiency.
  • Positioning the organisation so that it can sustain quality standards during periods of change.
  • Ensuring prompt and effective responses to changes in the expectations and needs of customers.

Organisations that adopt quality management systems assume that everyone in the organisation impacts on the quality of services or products. It is generally recognised that in all but the

smallest  organisations,  prompt  responses  to  change  depend  on  flattening  organisational

Quality Management Systems for ETQAs                                                                                                            9

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

hierarchy and delegating as much responsibility as is possible to those who directly deliver the service(s). Effective responses depend on a clear understanding of what is important to the client, what the required standard is, and ownership of the requisite skills and knowledge.

Quality Assurance

Quality assurance refers to the sum of activities that assure the quality of products and services at the time of production or delivery. It includes:

  • Clarifying and describing accurately and comprehensively what the customer expects and needs.
  • Ensuring that those who make the product or deliver the service have a clear, comprehensive and accurate understanding of the quality standard.
  • Ensuring that those who make the product or deliver the service have available resources and systems that can deliver the required quality.
  • Ensuring that those who make the product or deliver the service have the skills, knowledge and motivation to make the products or deliver the service.
  • Ensuring that those who make the product or deliver the service have the means and skills to monitor the quality of what they make or deliver and to modify what they do to better meet the required standard.
  • Independently auditing and monitoring quality and feeding back this information to those who produce or provide or are otherwise in a position to contribute to enhancing quality.

Quality assurance procedures are frequently applied only to the activities and products associated directly with the goods and services provided to external customers.

Quality Audits

Quality Auditing is part of, but not the totality of, a Quality Assurance System. Quality audits are activities undertaken to measure the quality of products or services that have already been made or delivered. Where a product or service has a number of components, each component may be subject to an audit. The findings of such an audit could contribute to achieving the desired quality end product or service. However, it is the decisions taken in response to the findings of the audit that influence the quality experienced by the customer, either with respect to that which has been audited or with respect to future products or services. Often different people  take  these  decisions  to  those  who  carry  out  the  audit;  indeed  it  is  generally recommended that it should be so. Therefore, in itself a quality audit has no impact on quality.

Quality Control

A Quality Audit only controls quality when the findings are used to decide whether or not a product is delivered to the customer. An audit cannot control the quality of a service because the audit can only happen while the service is being delivered or afterwards. At best an audit may influence the quality of services in the future.

The person who makes the product or delivers the service controls quality ultimately.

Even where an audit is used to decide if a product is fit to deliver to the external customer, the

10                                                                                                        Quality Management Systems for ETQAs

Quality Terminology      2

person or people who make the product control the quality of goods that are delivered to the auditor: the internal customer.

Note 1: A Note about Quality and Cost

In many  contexts  quality  management  systems  are often  regarded  as a zero  cost  item.  The outcome of an effective quality management system means that an organisation provides products or services that consistently  meet or exceed customers  expectations,  even as the expectations and needs change. As a result, the organisation will be able to recover the cost of the QMS through increased business, through reduced waste, reduced expenditure associated with pacifying dissatisfied customers and making good that which is unsatisfactory, and/or through being able to charge a higher price for increased added-value.

The means by which Providers of education,  training and qualifications  and standards,  or any other part of the NQF system will be able to recover costs is not clear.

Quality management systems thus will have a cost. The challenge is to create and operate quality

management systems that maximise effectiveness within the resources that are available.

Quality Management Systems for ETQAs                                                                                                         11

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

Chapter 3

Quality Management Systems

The achievement, maintenance and enhancement of quality depend on establishing an organisational culture that puts quality first. While the processes of quality assurance and auditing  are  important,  they  tend  to  deal  with  the  here  and  now.  On  the  other  hand, achievement of consistently high standards of quality, even when product and service specifications change, depends on establishing strong foundations and preparing for the future. A quality management system is concerned with assuring and maximising quality now and in the future.

As already noted, it is the person who delivers a service or makes a product that controls its quality. However, everybody employed in an organisation, directly or indirectly impacts on the quality of products or services as experienced by the ultimate customer: the external customer. Each team and employee has a customer for his or her activities, often within the organisation itself: an internal customer.

Thus, an important facet of a QMS is empowering and enabling each employee and team to deliver services or products of the quality required by their customers, internal or external, today, tomorrow, next week and next year. Given the pace of change, frequently this can only be effected if each employee is enabled and empowered to adapt practices and procedures to better  meet  new  customer  expectations  and  needs.  This  argues  for  minimising  what  is prescribed in order to avoid inhibiting desirable action.

On the other hand, each employee needs to be clear about who their customers are and what their needs and expectations are. This information needs to be readily available in a clear and easily understandable form. Further, as the external customers’ needs or expectations change, as the organisation introduces new technologies, materials and systems to better, more cost effectively and efficiently meet the external customers’ expectations and needs, so the needs of internal customers will change. Individual employees therefore need to know what is expected of them, and this information has to be made available in a way that can be quickly modified to reflect changes.

If employees are to be empowered and enabled to enhance the quality of the services and products that they create or provide, they need to know how well they are doing and what could be done better. They need regular, authoritative and constructive feedback on performance. This means that the quality of products and services delivered needs to be regularly monitored. Additionally, customer satisfaction needs to be regularly monitored in order to ensure that the quality standard continues to meet their needs and expectations.

12                                                                                                        Quality Management Systems for ETQAs

Quality Management Systems   3

Table  1: Characteristics of a Quality Management System

To summarise, a QMS:

  1. Identifies the customer or beneficiary (internal or external) for each role within an organisation and specifies the current standard that will meet or exceed the customer’s or beneficiary’s needs and expectations.
  1. Ensures all employees are empowered  and enabled to continually  contribute  to achieving the required quality standard.

All employees and teams within the organisation should know how their responsibilities affect product   or  service  quality   and  have  criteria   against  which   they  can  measure  their performances as they impact on quality. As part of this, each employee should know who his or her customers are (who benefits or depends on their activities) and what would constitute a high quality of service for them.

Each employee or team should be equipped  with the skills, knowledge  and resources of the necessary quality to be able to deliver products  or services of the required standard.

  1. 3. Ensures all employees are empowered and enabled to monitor their impact on quality and contribute to its

All employees and teams should be enabled and required to continually monitor their impact on quality (and be provided with independent audit information about their conformance  with the required standard), so that they can identify  where they could  contribute  to enhancing quality, and plan and take action toward that end.

Users’  perceptions   of  the  quality  of  services  received  should  be  reviewed  regularly. Providers of services should review feedback from users and identify how quality might be improved, and plan and act to improve performance.

  1. Creates and sustains a ‘quality’ culture.

The continual and consistent  achievement and maintenance of high quality standards under conditions  where clients’ expectations  are likely to continue to change depends on creating and supporting  a quality culture in the organisation.

Quality Management Systems for ETQAs                                                                                                         13

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

In essence, an effective quality management system establishes a quality spiral as standards are continually enhanced.

Diagram 1: Quality Spirals

14                                                                                                        Quality Management Systems for ETQAs

Chapter 4

A quality management system for the National

Qualifications Framework  (NQF)

In many ways, the NQF System is comparable to a large organisation, having a clear and shared purpose laid out in the Act. Within the “NQF Organisation” SAQA creates the vision, sets the policies, defines the timetable, delegates the tasks and defines quality of performance for those to whom they are delegated. It is the equivalent of the Board and senior executive of an organisation. The SGBs define the service standards in terms of the specific outcomes that should match the vision described by SAQA. The Providers are the powerhouses, the productive unit, the creators and constituent Providers of the service. The ETQAs have the quality audit and assurance role.

But what of the NSBs and the suggested Moderating Bodies? What is their role within this organisation? Their task is complex and diverse. As already discussed, the NQF entails providing services that are diverse in the content, as diverse as the customers for the service. By creating multiple NSBs, to act in essence as agents of SAQA, this diversity is managed. SAQA requires that each NSB include representation from the various stakeholders for the ultimate service. In this way, SAQA is seeking to ensure that the standards developed by individual SGBs reflect the vision, address the problems identified and meet the needs of the diverse stakeholders. Their role is to quality assure the standards setters.

The potential Moderating Bodies have a similar role. To simplify operations, Providers are to be served by a single ETQA, but the implementation of standards is not to be limited to a particular sector of Providers. (Indeed this would be contrary to two of the goals of the Act, the development of an integrated framework and enhanced mobility between the different parts of the system of Providers). So different ETQAs will be quality-assuring  services based  on  the  same  standards.  Mobility  and  credibility  will  depend  on  an  adequately consistent interpretation of the standards by all that use them. The role of the Moderating Bodies will be to assure this consistency across ETQAs. They too are agents of SAQA.

Table 2 examines the responsibilities of each part of the “NQF Organisation” in managing quality.

As will be noted, each body within the “NQF Organisation” contributes to the quality management system in a variety of ways. Through its consultations and publications, and through fostering wider debate, SAQA is fostering a quality culture. It is also enabling its partners to play their part through clarifying their roles and responsibilities and by helping

them to examine what that means in the context of the NQF.

Quality Management Systems for ETQAs                                                                                                         15

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

Table  2: Managing Quality in the  “NQF Organisation”

Responsibilities

SAQA                                 SGBs

Create and sustain a ‘quality’ culture throughout the “NQF Organisation”

Continually ask of itself and of all parts of the NQF Organi-sation “How are we doing, how could we do

better?”

Identification of customers and

beneficiaries

Defined by the Act

Interpreted by SAQA

Ensure the relevance, comprehensiveness and clarity

of the standards and qualifications

General expectation specified by the Act, interpreted by SAQA, especially regarding

breadth, nature and format

Define clear, comprehensive standards and qualifications that are fit for the various purposes defined by the Act

and SAQA

Ensure standard and qualifications is accurately and comprehensively understood

Provide general guidance about the standards and qualifications and their purpose and the wider goals of the NQF

Provide specific guidance regarding the purpose, interpretation and application of the standards and qualifications, associated

learning and assessment

Ensure facilitator of learning and assessment has relevant skills and knowledge to facilitate learning and

design sound assessments

Ensure that facilitators of learning and assessment have access to the requisite

skills and knowledge

Provide guidance on the skills and knowledge required by the facilitator of

learning and assessment

16                                                                                                        Quality Management Systems for ETQAs

4

A quality management system for the National

Qualifications Framework  (NQF)

Providers                           ETQAs                               NSBs                                  Moderating Bodies

Continually ask of themselves, “How are we doing, how could we do better? What do our customers require of us?

How can we ensure that they are getting what they need and expect from us?”

What do our ‘internal customers’ (SGBs, Providers, ETQAs, NSBs and Moderating Bodies) need of us?

How can we ensure they are getting what they need and expect of us?

Enable and empower employees to deliver services to or above the required or expected standard

Regularly set goals for new and improved services, design and deliver improved services that meet or exceed customers’ expectations

Help customers identify learning opportunities and qualifications and standards that match their needs and

aspirations

Contribute to assuring standards quality by collecting field information and providing feedback to

NSBs/SGBs

Assure standards and qualifications developed by SGBs match the expectations and needs

of SAQA

Ensure facilitators of learning and assessment are adequately briefed, receive the standards and qualifications in good time

and can accurately interpret

the guidance

Assure that standards and qualifications can be easily and accurately interpreted by facilitators of learning and assessment and that

comprehensive and useable

guidance is available

Ensure the facilitator of learning and assessment have the relevant skills and knowledge or can readily acquire them

Confirm that Providers ensure that facilitator of learning and assessment has requisite skills and knowledge

Ensure that SGBs provide clear, valid and comprehensive guidance on the skills and knowledge that facilitators of learning and

assessment will require

Quality Management Systems for ETQAs                                                                                                         17

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

Table  2 continued:  Managing Quality in the  “NQF Organisation”

Responsibilities                         SAQA                              SGBs

Ensure the facilitator of learning and assessment can and does monitor the effectiveness of learning and assessment strategies

Ensure relevant training of learning and assessment facilitator in technically sound assessment methods

is readily accessible

Ensure that resources of a suitable quality are available

Define standards and qualifications that are realistic given the resources that are, or can be made

available.

Provide guidance on the resources that are required to facilitate and assess

learning

Regularly assess customer satisfaction, report on it and identify where expectations and needs are not being met

Audit service quality and report on that is achieved

Regularly seek and receive feedback from Government and national representatives of key stakeholder groups

Regularly seek and receive feedback from ‘internal customers’: NSBs, Providers, ETQAs and stakeholders represented on the SGBs

18                                                                                                        Quality Management Systems for ETQAs

4

A quality management system for the National

Qualifications Framework  (NQF)

Providers                             ETQAs                               NSBs                        Moderating Bodies

Ensure the facilitator of learning and assessment has assessor skills and regularly monitors effectiveness of learning facilitation and

assessment activities

Confirm that Providers ensure that the facilitator of learning and assessment regularly monitors and reports on effectiveness of learning and assessment

activities

Ensure that the facilitator of learning and assessment modifies practices when desirable, in the light of

past effectiveness

Confirm that Providers ensure that practices are enhanced in the light of what is learned from

monitoring activities

Ensure the facilitator of learning and assessment knows and understands the resources necessary to achieve the required

standard and qualifications.

Ensure standards and qualifications are realistic given the resources that are, or could be made, available

Provide those resources that fall within the responsibility of the

Providers

Confirm that suitable resources are available and are used to good effect

Ensure clear and adequate guidance on resources required is made available

Regularly seek, receive and evaluate feedback from learners for qualifications and standards and their

sponsors

Regularly seek, receive and act on feedback from their

‘internal customers’: SAQA, Providers, NSBs/SGBs and

stakeholders

Regularly seek and receive feedback from their ‘internal customers’: SAQA and stakeholders represented

on NSBs

Regularly seek and receive feedback from their ‘internal customers’: SAQA and ETQAs

Establish and operate internal audit processes

Monitor Providers outcomes and internal audit processes. Report to Providers, SAQA

and NSBs/SGBs

Monitor ETQAs and report to SAQA and give feedback to ETQAs

Quality Management Systems for ETQAs                                                                                                         19

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

Chapter 5

The ETQA’s role in the quality management system for the NQF

Regulations R1127, under the South African Qualifications and standards Authority Act, 1995 specifies the roles and responsibilities of ETQAs. They are accredited by SAQA for “the purpose of monitoring and auditing achievements in terms of national standards or qualifications and standards”.

Their quality management functions are to:

  1. Accredit  Constituent  Providers  (subject  to  them  having the  capacity  to  deliver relevant learning and assessment opportunities) for specific standards or qualifications and standards.
  2. Promote quality amongst Providers.
  3. Monitor provision by Providers.
  4. Evaluate assessment (by Providers) and facilitate moderation amongst Providers.
  5. Register constituent assessors for specified standards or qualifications.
  6. Take responsibility for the certification of learners.
  7. Co-operate with Moderating Bodies appointed by SAQA.
  8. Recommend  new  standards  or  qualifications  to  NSBs,  or  modifications  to  existing standards.
  9. Maintain a database acceptable to SAQA.
  10. Submit reports.
  11. 1 Perform such other functions as may be assigned by SAQA.

While an ETQA can delegate some of these functions, for example to providers, they cannot delegate accountability.

Of the 10 functions specified in detail and that contribute to quality management, only two, functions 3 and 4, directly pertain to auditing and monitoring quality as specified in the Regulations. One might speculate that this is a practical compromise arising from the recognition that auditing and monitoring are amongst the least cost-effective methods of assuring and securing the required quality but probably have the highest level of credibility in the eyes of the public. Nevertheless, it is also important to keep in mind that the Regulations recognise that the ETQAs will influence quality in a variety of ways including but not solely through auditing and monitoring.

Table  2  identifies  55  ways  in  which  the  bodies  that  make  up  the  “NQF  Organisation” contribute to achieving the desired quality. Only 10 of these fall under the ETQAs. The roles and responsibilities given to the ETQAs’ are critical in securing the desired quality but so are the remaining 45 that fall to their partners: every part of the “NQF Organisation” contributes to securing and assuring the quality of learning opportunities and qualifications and standards.

20                                                                                                        Quality Management Systems for ETQAs

5

The ETQA’s role in the quality management system for the NQF

The system will be only as good as the weakest link.

Table 2 reviews the roles that contribute to quality from the most to the least cost effective. Those attributed to the ETQAs are listed in Table 3 in order of declining cost-effectiveness.

Table  3: The Quality Management Roles  of ETQAs within the  “NQF System”

  1. Create and sustain a quality culture.
  2. Contribute to ensuring  the  relevance,  comprehensiveness  and  clarity  of  the  standards  and qualifications.
  3. Confirm that Providers ensure that the facilitators of learning and assessment have the requisite skills.
  4. Confirm that Providers  regularly  monitor  and  report  on  the  effectiveness  of  learning  and qualifications and standards.
  5. Confirm that the Providers ensure that practices are enhanced in the light of what is learned from monitoring activities.
  6. Confirm that suitable resources are available and are used to good effect.
  7. Regularly seek, receive and act on feedback from their customers’:  SAQA, Providers,  NSBs, SGBs, moderating bodies and stakeholders.
  8. Monitor Providers outcomes and internal audit process and report back to Providers, SAQA and

NSBs.

Each of the above divides into a number of activities. These are described in Table 4 and discussed in the text following.

A fundamental role of the ETQA is to assure the quality of the services made available by

Providers. There are three basic processes involved in quality assurance in this context:

  • Accreditation, through which the Provider satisfies the ETQA that it is able and willing to deliver services of the quality required.
  • Monitoring, by the Provider, of the quality achieved and reporting this information to the

ETQA.

  • Auditing, through which the ETQA assures the integrity and accuracy of the submitted reports. This will entail the detailed scrutiny of the reports and visits to the Provider to check their authenticity and accuracy.

Accreditation, monitoring and auditing are likely to be annual events, at least initially. After the first year, re-accreditation will be closely linked to the receipt of reports from the Provider and any audit of the Provider that the ETQA has conducted.

Through the accreditation process the ETQA will want to be assured that:

  • The Provider’s quality aspirations address each of the quality indicators and are both ambitious and yet realistic.
  • The Provider has, or will put in place, systems to collect sufficient, authentic, current, and valid evidence pertaining to the quality achieved.
  • The Provider has or will ensure that all of the resources necessary to meet the quality expectation (including staff competent in learning facilitation and assessment) are or will

be available.

Quality Management Systems for ETQAs                                                                                                         21

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

The ETQA and Provider will also agree what information is to be reported by when and how. An ETQA may wish to establish standardised forms to be used by Providers for reporting achievements for each indicator. They will also require information about progress towards establishing and maintaining a quality culture within the Providing organisation, for example the achievement of the Investors in People award or ISO 9000:2000.

The current quality indicators are based on the objectives of the NQF, for both qualifications and programmes, that Providers:

  • Use the standards and integrate theory and practice.
  • Utilise suitable learning and assessment processes for the prescribed learning outcomes.
  • Better enable individual learners to contribute to the reconstruction and development of the country and the individual’s social-political-economic development.
  • Facilitate and enhance access, mobility and progression.
  • Redress previous inequities, particularly making available opportunities for those who could not previously access them.
  • Periodically collect, store and report information describing achievements for each of the other indicators.

Providers are expected to continually monitor their achievements against the goals they have agreed with the ETQA, for each indicator. As noted above, these achievements will be formally reported to the ETQA each year.

The process of auditing is considered in more detail later in this Section. It is suggested that the most recent report by the Provider will be the primary focus for an audit although the ETQA may also wish to review evidence of achievement against the indicators for other reasons; for example in response to SAQA’s current agenda and the needs of the SGBs.

In  conformance  with  their  developmental  role,  ETQAs  will  want  to  ensure  that  a Provider’s goals for assuring and enhancing quality are ambitious, but realistic and adequately address all indicators of quality. The annual accreditation process will probably build on the Provider’s report of achievements. They may entail negotiations through which the ETQA will foster the inculcation of a quality culture within the Providing organisation. The ETQA will

provide feedback on where improvements are necessary or desirable.

22                                                                                                        Quality Management Systems for ETQAs

5

The ETQA’s role in the quality management system for the NQF

Table  4: Activities Associated with the  Quality Management Roles  of ETQAs

  1. To create and sustain a quality culture:
  • ETQAs should continually ask and address the following questions:

“How well are we contributing  to assuring the quality of learning opportunities  and qualifications and standards  within the NQF, through  our auditing, monitoring  and feedback  activities?  How could we do better?”

“What do our customers (SAQA, Providers, NSBs, SGBs, Moderating  bodies and stakeholders) require of us? How can we better ensure that they are getting what they need and expect from us?”

  • An ETQA should establish a service contract with each of its customers:  SAQA, NSBs, SGBs, moderating bodies, stakeholders and Providers.
  • An ETQA should define what it requires of each role within its organisation and set standards of performance that link to the customers’ expectations  and needs from its service contracts.
  1. To contribute to ensuring the relevance, comprehensiveness and clarity of the standards:
  • An ETQA must establish a working relationship with its Providers such that they (the Providers) are motivated to provide the requisite information in a timely and well argued and ordered fashion.
  • ETQAs will require facilitators of learning and assessment to have in place well considered  and effective mechanisms to regularly collect and review how well their programmes and qualifications and standards meet existing and potential learners’ needs and mechanisms whereby facilitators of learning and assessment or assessors can register apparent inadequacies in the standards.
  • Providers will be required to periodically report information about inadequacies in standards and qualifications and regarding the need for new standards  to the ETQA, as part of their regular feedback activities.
  • ETQAs will need mechanisms to check why standards are not met, when that is the case.
  1. To confirm that Providers ensure that the facilitators of learning and assessment have the requisite skills:
  • Regularly require Providers to make a formal statement confirming that facilitators of learning and assessment and assessors are competent as defined by the relevant SGB.
  • ETQAs will check the authenticity of such statements during audits.
  1. To confirm that Providers regularly monitor and report on the effectiveness of learning and assessment activities:
  • It is envisaged that ETQAs will require (as part of an Accreditation Agreement) Providers to make a clear statement of how quality will be regularly monitored and reported.
  • ETQAs will want to assure themselves of the validity and sufficiency  of both  reports  and the constituent data, and that the process of collecting and analysing the data has been undertaken

diligently and with integrity.

Quality Management Systems for ETQAs                                                                                                         23

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

Table 4 continued

  1. To confirm that the Providers ensure that practices are enhanced in the light of what is learned from monitoring activities:
  • Providers should share with the ETQA their quality enhancement achievements and goals. This information would be reviewed and checked by ETQAs for its validity and currency at the time of auditing and monitoring visits.
  1. To confirm that suitable resources are available and are used to good effect:
  • ETQAs should invite Providers  to describe  how  relevant resources  will be made available to learners and candidates.
  • The ETQAs will ensure that the access-to-resources requirements of standards are met at the time of accreditation  and during subsequent audits or monitoring activities.
  1. To regularly seek, receive and act on feedback from their ‘internal customers’: SAQA, Providers, NSB, SGB and stakeholders:
  • ETQAs will ensure that their reports provide all of the information required by SAQA in a form that is easy to use and interpret accurately.
  • Similarly with regard to periodic feedback to NSBs, the ETQA should check on the usefulness of what is reported and seek advice as to how its value might be enhanced.
  • The requirements of Providers regarding reporting will be described clearly in the Accreditation

Agreement and will be subject to regular review.

  • ETQAs will  provide   regular,  constructive   feedback   to  Providers  regarding  the  utility  and substance of reports received, and help them minimise the work required to produce them.
  1. To monitor Providers  outcomes  and  their  internal  audit  process  and  report  back  to

Providers, SAQA and NSBs:

  • ETQAs may audit Providers for one, or a combination, of reasons. Six reasons are identified on page 29.
  • ETQAs will assure the quality of external exams or tests during the design phase and their completion. Six steps are identified on page 30.
  • ETQAs will  assure  the  quality  assessments  that  are  designed,   delivered  and  evaluated

institutionally. Six steps are identified on page 31.

Where a Provider does not meet the required standards, there is scope for provisional accreditation for up to two years, subject to the Provider implementing a development programme that is agreed with the ETQA, and that addresses the issues of concern. Under these conditions it is incumbent on the ETQA to provide what assistance it can to help the Provider identify priorities for enhancing quality, and suggest what it might need to do.

Where an ETQA considers that the performance of a Provider, or its progress in enhancing performance, is unsatisfactory with respect to any aspect of quality, they would need to consider

if the Provider should no longer be accredited.

24                                                                                                        Quality Management Systems for ETQAs

5

The ETQA’s role in the quality management system for the NQF

A: Create and sustain a quality culture

A set of questions has been identified, the answers to which should inform the two activities:

‘Enabling and empowering employees to deliver services to or above the standard required’ and ‘Regularly set goals for new and improved services’. Applying them to the ETQA context, the questions can be expressed as follows:

“How well are we contributing  to assuring the quality of learning opportunities and qualifications and standards within the NQF, through our auditing, monitoring and feedback activities? How could we do better?”

“What do our customers (SAQA, Providers, NSBs, SGBs, Moderating  bodies and stakeholders)

require of us? How can we better ensure that they are getting what they need and expect from us?”

The answers to these questions should enable an ETQA to regularly set itself new goals that represent better standards of service. The answers should enable an ETQA to identify where performance within its organisational structure could be improved and take steps to secure improvements through enabling and empowering its employees to do better.

The questions are deceptively simple. The challenge is collecting the data that accurately reflects  how well the organisation  is doing  in each  of the activities  associated  with the

remaining quality responsibilities discussed below.

Ideally an ETQA should establish a service contract with each of its customers: SAQA, NSBs, SGBs, moderating bodies, stakeholders and Providers.

A service contract specifies precisely but comprehensively and clearly the services that will be provided, to whom, when and to what standard.

Given that each ETQA will need to establish a service contract with SAQA and with the NSBs, there would be value in a group of ETQAs negotiating a standard contract with SAQA and with a representative group of NSBs/SGBs. SAQA might take the lead, bringing together representative groups to facilitate the drafting of these service agreements. The service contract with SAQA would reflect the overarching responsibility of SAQA and the expectation that it will have to produce regular reports to Ministers and others on progress in implementing the NQF.

It will also reflect the ETQA quality assurance role. Similarly, ETQA relationships with the NSBs/SGBs also reflect quality assurance responsibilities.

Given the diversity of the relationship between each ETQA and its Providers the service contract between ETQAs and Providers is likely to vary somewhat. However, it is likely to deal with most aspects of their quality management relationship.

It may well be that the service contracts identified will cover all stakeholders. On the other hand, the Professional Bodies and SETAs have other accountabilities. They would be well advised to establish service contracts with these other groups to whom they account, to ensure

that all their responsibilities are equally well defined.

Quality Management Systems for ETQAs                                                                                                         25

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

An ETQA should define what it requires of each role within its organisation and set standards of performance that link to the customers’  expectations  and needs from its service contracts.

Dependent on the criticality of tasks associated with these performance standards, performance data might be collected monthly, quarterly or annually, preferably, primarily by those undertaking the activities. Each employee should be encouraged to review performance periodically (for example, quarterly), note and celebrate improvements, and set targets for enhanced performance, including identifying learning needs and how they will be met.

These procedures not only have the capacity to establish a quality culture in the ETQA, but will provide substance for two of the quality management requirements identified in the Regulations: 9 “Maintain a database acceptable to SAQA” and “Submit reports”.

B: Contribute to ensuring the relevance, comprehensiveness and clarity of the standards

ETQAs’ prime responsibility in this context is the provision of field intelligence to the NSBs in conformance with quality management “Recommend new standards or qualifications to NSBs, or modifications to existing standards and qualifications”.

An ETQA must establish a working relationship with its Providers such that they (the Providers) are motivated to provide the requisite information in a timely, well argued and ordered fashion.

Where an intermediary is involved, an ETQA must ensure that it establishes a similar relationship with Providers. This is particularly pertinent to the identification of new standards and qualifications and standards that may be needed.

Indeed Providers who have a quality culture are interested in what their learners feel about

the services received.

ETQAs will require facilitators of learning and assessment to have in place:

  • Well-considered  and  effective  mechanisms  to  regularly  collect  and  review  how  well  their programmes and qualifications and standards meet existing and potential learners’ needs.
  • Mechanisms whereby facilitators of learning and assessment can register apparent inadequacies

in the qualifications and standards.

For clients of education and training, the most important characteristics of quality are usually:

  1. Relevance (of what is learned to the purpose of learning).
  2. The learning process (the process enabling the individual to learn).
  3. Accessibility (having in mind the learner’s circumstances).
  4. Currency (having regard to the purpose for learning).

26                                                                                                        Quality Management Systems for ETQAs

5

The ETQA’s role in the quality management system for the NQF

For qualifications, in addition to relevance and accessibility, reliability and currency are the important characteristics of quality. In addition to information about the relevance and clarity of the standards, learner surveys will collect information about many other facets of their learning and assessment experiences as considered below.

Providers will be required to periodically  report information  about inadequacies  in standards  and regarding the need for new standards, to the ETQA, as part of their regular feedback activities.

The inadequacies registered by facilitators of learning and assessment may relate to their perceptions of learners’ needs and expectations or their own experiences in attempting to use the standards.

In the event that providers are delivering similar qualifications and standards, an issue that may need  to be considered,  regarding  possible  new standards  and qualifications,  is that ETQAs should check such providers to determine if they perceive a similar need. There is no doubt, if such a need is identified by a number of Providers, the development of the requisite standards and qualifications may assume a higher priority. On the other hand, if Providers are in competition, it may be that the matter should be handled in a confidential manner so that

the Provider who has identified the need retains a market advantage.

ETQAs will need mechanisms to check why standards are not met, when that is the case.

In addition to representations from Providers, the need to modify existing standards and qualifications  may  be  noted  from  other  quality  management  activities.  For  example,  a failure to meet the required standards may be the outcome of poorly expressed standards, of standards  that  are  unachievable,  or  standards  and  qualifications  that  are  irrelevant  or otherwise do not motivate learners. This places a premium on ETQAs having mechanisms to investigate why a standard has not been met. Ideally, and in keeping with a quality culture, such investigations would be undertaken with the full collaboration of the Provider and in a way that is constructive rather than punitive.

C: Confirm that providers ensure that the facilitators of learning and assessment have the requisite skills

Ensuring that those who facilitate learning and who assess individuals for the purpose of awarding qualifications are competent  (as defined by the SGBs/NSBs or SAQA when relevant) is primarily the responsibility  of the Provider. Confirming that the Provider is meeting its responsibility  is the role of the ETQA.

Quality Management Systems for ETQAs                                                                                                         27

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

It is likely to be confirmed, initially, through the accreditation process: “Accredit Constituent Providers” and subsequently through auditing the Provider: “Monitor provision by Providers”. With respect to summative assessment, it is also a responsibility of ETQAs: “Register constituent assessors for specified standards or qualifications”.

In the past, if the competence of facilitators of learning and assessment was subject to external confirmation, it normally happened prior to the Provider gaining approval to run a specific programme. Additionally, if the problem was significant, Providers may have been required to notify the external body every time there was a change in the people involved and confirm their competence.

It is stated in the NSB Regulations that SGBs must provide criteria for the registration of assessors with regard to specific standards and qualifications. This implies that the criteria

would include skills and knowledge.

Providers will be required to make a formal statement that learning and assessment facilitators are competent,  ETQAs would probably check the authenticity  of such statements during audits.

ETQAs will need to decide, in the light of experience, if this is something that should be confirmed only once, periodically, or routinely. In any event, it is likely that the recommendations or specifications of the SGBs are being observed.

Perhaps more challenging, however, is that the Act foresees different learning outcomes that carry with them a need for different and new learning and assessment strategies. The facilitation  techniques  traditionally  used  to  facilitate  knowledge  transmission  from  the learning providers to learner, and the assessment techniques used to determine to what extent transmission had happened are being increasingly challenged in both this context and others.

There is general recognition that didactic teaching is not very effective at helping individuals  to  acquire  deep  (enduring)  knowledge.  To  secure  deep  knowledge,  learners require opportunities to apply the knowledge in contexts that are relevant to them. To acquire enduring and transferable skills, individuals require regular opportunities to practice them in diverse contexts. The regular provision of informative feedback further enhances learning.

Similarly, traditional tests or quizzes have limitations as assessment tools. At best they test knowledge or the application of basic skills (especially basic skills in reading, writing and numeracy).

At worst they assess the facilitator of learning and assessment’s ability to devise good tests and the learner’s ability to read the mind of the facilitator of learning and assessment. Increasingly there is recognition that if we want to know about an individual’s ability to use skills and knowledge in contexts outside of the classroom, we need different ways of assessing: authentic assessment. There is a steady trend towards assessing performance. This trend should not be taken to imply there is no value in traditional forms of examinations and tests. We just need to be more realistic about what they can and cannot tell us, especially with regard to abilities that need to be used outside of the classroom and examinations hall.

Too many facilitators of learning and assessment lack confidence, experience and knowledge of how to facilitate learning in this way, or how to develop skills, especially problem solving, teamwork and communication skills. A similar problem pertains to assessment. It is probable that

most ETQAs will come across these problems regularly.

28                                                                                                        Quality Management Systems for ETQAs

5

The ETQA’s role in the quality management system for the NQF

D: Confirm that providers regularly monitor and report on the effectiveness of learning and assessment activities

ETQAs are required to: “Accredit Constituent Providers” and “ Facilitate moderation amongst

Providers”.

For the success of the NQF, Providers must meet the objectives which are ensuring integration and the achievement of learning outcomes through maximising the effectiveness of teaching and assessment, enhanced access, mobility and progression, the personal and national development of the learner and the redress of past inequalities. Providers need to regularly determine how well they are doing in terms of the above objectives, and learners’ satisfaction with the services provided. Indeed they are in the best position to assure the quality of services.

It is envisaged that ETQAs will require (as part of an Accreditation  Agreement) Providers to:

  • Set goals for each objective that represents an improvement on those previously achieved.
  • Make a clear statement of how quality will be regularly monitored and reported with each objective.
  • Report progress against identified goals.

It has already been noted that this will include surveys of learners and that through these surveys, inadequacies in the standards and qualifications will be noted. In addition, it would be desirable for learners to be invited to comment on the following:

  • Initial advice and support when choosing a programme or qualification.
  • Effectiveness of learning opportunities.
  • Apparent validity and fairness of assessments.
  • Adequacy and helpfulness of feedback.
  • Availability of resources.
  • Enhancement of the learner’s personal and national development.
  • Redress of past inequalities.

Providers  should  also  collect  information  from  learners  to  learn  how  well  the  chosen programme or qualification helps the individual achieve her or his goals.

Ideally, a Provider would collect and analyse such information from all learners or from a significant sample. The frequency with which this information is summarised and passed to the ETQA may vary depending on the numbers of learners involved, the diversity of programmes and the qualifications and standards offered by the Provider and the ETQA’s

reporting responsibilities. However, it is likely to be required at least once a year.

ETQAs will  want  to  assure  themselves  of  the  validity  and  sufficiency  of  both  reports  and  the constituent  data, and that the process  of collecting  and analysing the data has been undertaken diligently and with integrity.

Quality Management Systems for ETQAs                                                                                                         29

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

The  submission  of  reports  on  a  regular  basis  would  provide  evidence  of  the  activity. Consistency in data and analysis would suggest diligence and accuracy but spot checks during auditing activities would be desirable.

Hopefully, over a period of a few years, a Provider would demonstrate a commitment to a quality culture and that it values and uses the data collected. Once an ETQA is persuaded of the commitment of the Provider to the effective monitoring of the quality of the services it provides, the focus of auditing must move to analysis rather than the need for the collection of data.

E: Confirm that the providers  ensure that practices  are enhanced in the light of what is learned from monitoring activities

Collecting and analysing data will not of itself assure or enhance the quality of a Provider’s services. It is the dissemination of that information to facilitators of learning and assessment and its use to plan and secure changes in practice that can lead to improved quality.

Providers should share with the ETQA their quality enhancement achievements and goals for each objective. This information would be reviewed and checked by ETQAs for its validity and currency at the time of auditing and monitoring visits.

ETQAs are required to “Promote quality amongst Providers. It would make most sense if Providers reports were linked to goals identified in previous reports, feedback from the ETQA subsequent to auditing and monitoring activities and the outcomes of learner surveys.

Again, the frequency with which this information is summarised and passed to the ETQAs may vary depending on the number of programmes/qualifications and standards involved, the diversity of programmes and qualifications and standards offered by the Provider and the ETQA’s reporting responsibilities. However, it is likely to be required at least once a year.

F: Confirm that suitable resources are available and are used to good effect

Along with C and D above, access to suitable resources is likely to be confirmed initially as part of the accreditation process (1. “Accredit Constituent Providers”).

The  ETQA’s  responsibility   will  be  to  ensure  that  resources  required  for  the  standards   and qualifications are met and there is access to those resources at the time of accreditation  and during subsequent audits or monitoring activities.

30                                                                                                        Quality Management Systems for ETQAs

5

The ETQA’s role in the quality management system for the NQF

In the past, where it has been the practice to accredit Providers, the availability of resources has been a major consideration, often examined in more detail than the competence of the facilitators of learning and assessment. The situation is changing, however.

Knowledge, technology and systems are advancing quickly, and their application is becoming more diverse and specialised. As a result, it is becoming increasingly difficult for Providers to sustain the currency of libraries and equipment. Indeed, in many instances it is just not possible, because financial resources are not available. Often, what are required are, on the one hand, the opportunity to develop basic skills and on the other, real-life learning experiences. The former may indeed require access to workshops and laboratories but the latter is more likely to be provided directly through assignments that draw on real-life.

More than ever before, we are recognising that there are increasingly diverse opportunities to learn, for example through the use of Internet and from daily experiences (that may be identified and created by the Provider).

As with learning facilitation and assessment skills, (C above), it is stated in the NSB

regulations that the SGB should provide guidance on the resources that need to be available with regard to specific standards and qualification.

ETQAs should  invite Providers  to  describe  how  relevant resources  will be made  available to learners at the time of initial accreditation.

It may be that the Provider will not have the specific resources but plans to access them in some way, for example, through work placements. The effectiveness of these arrangements can be monitored in a number of ways such as:

  • The Provider should regularly seek confirmation from learners that adequate resources were available (D above).
  • As part of auditing and monitoring activities undertaken by the ETQA.

G: Regularly seek, receive and act on feedback from their

‘internal customers’: SAQA, providers, SGB, NSB and stakeholders

ETQAs are required to “Co-operate with Moderating Bodies appointed by SAQA”, “Maintain a  database  acceptable  to  SAQA”,  “Submit  reports”,  “Provide  feedback  to  NSBs”  and “Promote quality amongst Providers”. In each instance, the two-way flow of information is critical to achieving and sustaining high quality standards.

In essence the Moderating Bodies are agents of SAQA. They are charged with ensuring that

ETQAs, who accredit programmes and standards for the same standards, assure that adequately comparable standards are applied across them all.

It is incumbent on ETQAs to ensure that their reports provide all of the information required by SAQA

in a form that is easy to use and interpret accurately.

Quality Management Systems for ETQAs                                                                                                         31

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

In addition to feedback received pertaining to the findings of Moderating Bodies, ETQAs will probably receive feedback about the utility and substance of reports submitted to SAQA. If there are inadequacies in the substance of such reports, it may mean that the ETQA will have to re-visit its analysis and interpretation of data collected, review its auditing and monitoring procedures, or revise its service agreement (Accreditation Agreement) with providers.

Similarly with  regard to periodic  feedback  to NSBs and SGBs, the ETQA should  check  on the usefulness of what is reported and seek advice as to how its value might be enhanced.

It is always easy to provide too much information. It is suggested that ETQAs continually seek feedback regarding the relevance and usefulness of the information they provide in order to identify ways in which reporting can be undertaken in a more concise and focused way.

Feedback  from  Moderating  Bodies,  SAQA or NSBs  may  suggest  that  more,  less  or

different information needs to be collected from Providers.

It is incumbent  on the ETQA to provide regular, constructive  feedback  to Providers regarding the utility and substance  of reports received, and help them minimise the work required to produce them.

ETQAs may identify ways in which the reporting of information by Providers might be made more effective and less time consuming for both parties.

It is suggested  that needs and expectations  are described  clearly in the Accreditation  Agreement and that they are the subject of regular review in order to better meet the needs of the ETQA and to reduce the work required of the Providers in their preparation.

The sector stakeholders will vary between ETQAs, where responsibilities go beyond those dictated for the “NFQ Organisation”; it is suggested that they should put in place similar systems to collect feedback from stakeholders.

H: Monitor constituent providers outcomes and internal audit process and report back to constituent providers, SAQA, NSB and SGB

ETQAs are charged with “Promoting quality amongst Providers”, “Monitoring provision by Providers”,  “Evaluating  assessment”,  and  “Taking  responsibility  for  the  certification  of learners”.

As already noted, auditing, in itself, is often one of the least cost-effective methods of managing quality and yet it is the process that often has the highest public visibility and credibility. Auditing, in the education and training context, assumes many different forms

depending on its purpose.

32                                                                                                        Quality Management Systems for ETQAs

5

The ETQA’s role in the quality management system for the NQF

ETQAs may audit Providers for one, or a combination,  of the following reasons. To collect evidence about:

  • The systems put in place by a Provider to assure the quality of its services.
  • Learning processes, Provider’s competence and resources.
  • Assessment of learning by Providers of learning opportunities.
  • Assessment undertaken by external bodies (examinations and qualification awarding bodies).
  • What has been learned, achieved, or is already known, through assessing the learner.
  • The degree to which access, mobility, progression and redress have been enhanced.

Auditing for the first three purposes has already been considered (see B-G above). Along with initial accreditation, audits for these purposes can be cost efficient. If they are supported by constructive feedback and a quality culture within the Provider organisation they can be very cost effective.

The effectiveness of assessment of learners (whether by the ETQA or intermediaries, such as an examination body, as a quality assurance mechanism, or by the learning Providers) depends on the technical soundness of the assessment. That is, it depends on the validity, sufficiency, currency and authenticity of the evidence collected and the consistency with which evidence is judged.

When assessment is used to contribute to assuring the quality of learning opportunities it provides a measure of how well the learning experience has prepared individuals for the assessment as well as what the learner may know and be able to do.

There is some concern in the assessment community as to how to assess, which is partially stimulated by fundamental questions regarding the purpose of learning. There is a growing acknowledgement that the demonstration of acquired knowledge and skills provides no guarantee of an individual’s ability to apply them in diverse contexts and that the learning of knowledge and skills through their application results in far deeper learning. This and other concerns have stimulated greater interest in assessing the ability to apply knowledge and skills in contexts that approximate real life.

Generally, the quality of assessments has been assured by reference to their content validity,

fairness and reliability (from a psychometric perspective).

Assessments  that  are designed,  delivered  and  evaluated  at  institutional  level,  may  be  quality assured by a selection from the following:

  • The provision of a detailed description of the required learning achievement.
  • Evidence of the competence of the assessor (as an assessor).
  • The provision, by the Providers, of assessment instruments for prior approval.
  • The submission of the evidence of learning generated by the learners (normally after it has been assessed institutionally) for the scrutiny of the qualifications and standards body or its agents.
  • The establishment of an internal quality assurance process within, by the Providers.
  • The regular or periodic scrutiny of the evidence of learning generated by the learners and the quality assurance process operated by the Providers and by a moderator.

Quality Management Systems for ETQAs                                                                                                         33

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

As noted previously, a few systems have made use of a combination of institutionally and externally designed assessments or the institutional application and evaluation of externally designed assessment instruments. Where a combination at institutional level and external assessment has been used, the results might be combined to generate a grade or one might be used to confirm the other.

There is growing interest in assessing performance, especially effective performance, in  work  contexts.  This  may  be  to  recognise  occupational,  professional  competence, especially where the ability to practice an occupation or profession is subject to some form of regulation. Similar assessments of occupational, professional competence have been adopted in countries that are concerned with facilitating mobility in the labour market, with upgrading the skills of the nation’s work force or with quality assuring employment related training. Such assessments have also been adopted in employment sectors where there is a desire to recognise competence, motivate employees to learn or enhance standards of performance. Professional bodies and those concerned with assessing occupational, professional competence are increasingly using a mixture of assessments: knowledge and skill  assessments  are  used  to  complement  assessment  by  Providers  of  education  and training  and  are  followed  by  a  period  of  assessment  on-the-job  before  gaining  full

recognition. This approach is considered in more detail in the next section.

34                                                                                                        Quality Management Systems for ETQAs

5

The ETQA’s role in the quality management system for the NQF

Summary

ETQAs, with SAQA, play a pivotal role within the quality management system of the NQF system. They can foster a quality culture within the NQF system both through creating such a culture within their own organisation and through helping and encouraging Providers to do the same.

However, if they are to foster a quality culture, they will depend on the other parts of the

NQF system (SAQA, NSBs/SGBs and Moderating Bodies) to play their part and will need to help them to do so.

In order to foster a quality culture amongst Providers ETQAs are likely to use a combination  of:

  • Initial accreditation  that  requires  Providers  to  undertake  a range  of  quality  assessment  and assurance activities and to report on the outcomes of these activities.
  • The monitoring activities  that  include  the direct  auditing  of reports,  systems,  processes  and outcomes and the judicious use of technically sound external assessment.
  • The use of technically  sound  assessments  to  confirm  the knowledge  and skills  of  learners, especially where it is to inform high-stakes decisions.
  • Annual reviews that include establishing and agreeing with Providers, new quality goals, taking

into account the previous year’s achievements and the outcomes of monitoring activities.

Quality Management Systems for ETQAs                                                                                                         35

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

Chapter 6:

Variations in the contexts in which the ETQAs Serving different sectors will operate that may impact on their QMS

ETQAs have been identified in two sectors: education and training sub-systems and economic. However, the latter appears to be dividing into two sub-groups industry/commerce sectors and the professional sector. Each of these groups is considered below. The observations are summarised and compared in Table 5.

A: Education and training sub-system ETQAs

Education and training sub-system ETQAs will generally have significantly larger Providers, and many more learners for qualifications and standards than will be the case for ETQAs in the economic sector.

The  purposes  for  learning  that  will  be  served  by Providers  accountable  to

Education and training sub-system ETQAs are likely to fall within the first six purposes identified in appendix 1.

  1. Seek and find fulfillment in life.
  2. Contribute to the  stability  and  growth  of  society  and  the  well  being  of  all  members  of  a community.
  3. Progress in the field of education.
  4. Respond effectively to future situations and expectations.
  5. 5. Meet his/her own economic needs and to contribute to the economic  survival and growth of a community  throughout  her/his working
  6. Gain employment in specific sectors of employment.

The nature of the learning outcomes will be such that they lend themselves to assessment through classroom-based assignments (especially purposes 1-3) and (for basic skills) tests. Purposes 4-6 are increasingly likely to require learning and the collection of evidence for assessment, away from the classroom. There will be considerable diversity in learning outcomes, content and the expertise of facilitators of learning and assessment.

If one  drew  a comparison  with  the  industrial  settings  in which  quality  management systems are well established, education is the equivalent of a high-volume business, participation in it is a long-term investment. Because it is not clear exactly how the learner will benefit economically from what is learned or from the qualification gained and because of the high-volume nature of the sector, quality management systems will tend more towards

the lower cost per individual learner than they would in the economic sector.

36                                                                                                        Quality Management Systems for ETQAs

6

Variations in the contexts in which the ETQAs Serving different sectors will operate that may impact  on their  QMS

To be cost effective, ETQAs in this sector may need to delegate responsibilities to Providers. They, in turn, may delegate responsibilities within the organisation. Responsibilities may also be delegated to examinations  bodies  for the testing  of basic  knowledge  and skills where required,  or for quality assuring Providers’ assessments of basic skills and knowledge.

The development of a quality culture within Provider’s organisations will be of critical importance. ETQAs in this sector can only audit the achievements of a small sample of learners for qualifications and standards. They will be very dependent on the Providers’ internal quality audit systems and the effectiveness with which Providers establish a quality culture within different parts of the organisation. Indeed, Providers may have to further delegate quality assurance to departments or schools and establish systems to ensure that there is systematic and effective quality assurance through internal reporting and auditing mechanisms, perhaps using an Institutional Research team to collect learner satisfaction information.  The  role  of  Quality  Manager  may  be  given  to  a  senior  member  of  the executive,  or  assumed  by  the  head  of  each  division  within  the  organisation,  reporting directly to the head of the organisation.

Where learning occurs outside of the classroom, the Provider may look to others to contribute to learning and to the collection of evidence of what has been learned. ETQAs will be interested in how the quality of these learning experiences and the evidence collected is

assured by the Provider.

ETQAs may need to phase in initial accreditation  requirements regarding learning and assessment facilitator competence  over an extended period of time.

Because of the nature of what has to be learned, there is a strong argument for facilitators of learning and assessment trained to be effective assessors as well as learning Providers. In particular, a large scale and ongoing investment in helping these learning Providers to be better able to facilitate the development of, and assessment of, skills such a problem solving, team working and oral communication may be required.

ETQAs serving this sector are likely to be large, have specialist expertise in learning and assessment as well as quality assurance and a formal internal quality management system led by a designated quality manager.

Because of the large number of learners (the high volume of the market), one would expect a high level of technical professionalism with regard to learning and assessment amongst the officials of education and training sub-system ETQAs. Further, because the ETQAs in this sector are likely to be relatively large, the establishment of a formal internal quality management system will also be of critical importance. For example, all roles and responsibilities need to be clearly defined, the customer (internal or external) for each should be identified and service standards specified comprehensively and in detail. Again, reflecting on the probable size of these ETQAs, there will

Quality Management Systems for ETQAs                                                                                                         37

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

need to be well established and managed systems for evaluating how the organisation is doing and how individual employees are performing, with methods for celebrating successes and setting goals for future improvements.

It is advisable that education  and training sub-system  ETQAs provide specific models of QMS for their providers.

B: Economic sector ETQAs: Sector Education and Training

Authorities (SETAs)

SETA ETQAs will tend to work with smaller sized Providers and smaller numbers of learners per provider  than will be the case for education  and training  sub-system  ETQAs. The purposes  for learning  that  will  be served  by  their  Providers  are likely to  fall within  the  last  seven purposes identified in Appendix 1.

  1. Respond effectively to future situations and expectations.
  2. Meet his/her own economic needs and to contribute to the economic survival and growth of a community throughout her/his working life.
  3. Gain employment in specific sectors of employment.
  4. Gain employment in specific jobs/occupations.
  5. To be recognized as competent in an occupation or job.
  6. Improve performance in specific work roles or tasks.
  7. Learn new skills and knowledge applicable to specific work roles or tasks.

However, reflecting on the wider objectives of the NQF to enhance education and training progression and contribute to the full personal development of each learner and the social and economic development of the nation at large, many of their Providers will also be concerned with the following learning purposes:

  1. Contribute to the stability and growth of society and the well being of all members of a community
  2. Progress in the field of education and training

ETQAs, and their Providers, in this sector are likely to be relatively small, have sector specialist as well as quality assurance expertise, and an internal quality management system that reflects easier internal communication.

For learning purposes 4-10, much of the learning will take place in the workplace and in other, non-education contexts, partially at least through experiential learning. Workplace learning may be underpinned by learning in workshops and classrooms, but the most important evidence of what has been learned will come from its effective application in real life situations. Providing organisations are likely to be smaller and expertise will be more focused and less

38                                                                                                        Quality Management Systems for ETQAs

6

Variations in the contexts in which the ETQAs Serving different sectors will operate that may impact  on their  QMS

diverse than in the case of the education and training sub-system Providers, reflecting their sector-specific remit. They are unlikely to require a separate quality manager.

It will be easier in this sector  to estimate  the added  value of quality  management  systems  and therefore assure and demonstrate their cost-effectiveness.

Because the learning is closely linked to performance in the workplace, the added value to learner and employer can be determined with some degree of certainty, especially for purposes

7-10. Given the ability to determine the potential added value, there will be greater certainty about what would constitute a worthwhile investment.

The link between investment and return is not necessarily as clear for purposes 4-6, which are more concerned with long-term flexibility and adaptability, especially for the learner.

However, given the fact that the learning and assessment is in the context of a particular sector, it should be possible to identify in general terms the potential cost to an employer if employees are not flexible and adaptable in the future. Similarly, the learner will have a reasonable sense of how the acquisition of these skills will enhance current employability and

future promotability and thus the scale of the potential economic return.

Where Providers are associated with employment sectors that are seeking to install quality management systems it is unlikely they will experience a great deal of difficulty  in doing likewise. The ETQA will want to ensure compatibility  with the requirements of the NFQ system.

Increasingly, Providers in this sector may be part of, or associated with, organisations that have or are seeking to establish a quality culture and quality management system. As long as these are based on a sound understanding of the purpose for establishing a quality culture and for managing quality, and the organisations are fully conscious of the expectations of ETQAs, it is likely that such Providers will find little difficulty in putting in place relevant systems. It may be a case of the ETQA negotiating to ensure that its needs are met uniformly across the sector (to help it in its role and especially to meet the needs of SAQA and the NSB/SGB), rather than having to provide direction and guidance on good practice.

Where Providers are not associated with a sector that has embraced a quality management system, they are likely to see value in developing and implementing such a system as a role model for the sector (and to build up their ability to provide other services for the sector). SETAs in particular will be able to make use of this motivation. Given the relatively smaller size of Providers and the fact that the focus is narrower than in the education and training sub-system, many aspects of quality management will be easier. For example, designing and operating an effective internal quality management system, developing an internal quality culture, collecting feedback from all learners, and providing effective, constructive and timely feedback to learning

and assessment facilitators should all be easier than in the education and training sub-system.

Quality Management Systems for ETQAs                                                                                                         39

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

It is likely that much of the learning and assessment will take place in the workplace in this sector. ETQA quality assurance systems will need to take this into account.

Reflecting on the nature of the learning outcomes associated with purposes 6-10, it is likely that much of the learning and evidence of effective learning will happen in the workplace. Supervisors and line managers can and should play a role in coaching and assessing performance. Indeed if they can’t, how can they be effective supervisors? In some ways, learning and assessment in the workplace are easier to achieve for these purposes than they would be in education settings. As long as there are clear standards for the products or services generated in the workplace, a learner can see the direct outcome of inadequate performance. (This is a critical contributor to reflective learning, which is the most efficient learning). A coach/assessor has a clear and easily recognized outcome against which to judge performance. It is often easier and less expensive to train workplace supervisors as assessors, and there can be spin-off benefits for the employer, not least through the improved performance of the supervisor.

However, workplace supervisors normally have a wide range of other duties that contribute to business success. Putting the initial responsibility for the collection and assessment of evidence on the shoulders of the learner can reduce the demands on the supervisor. Techniques associated with the recognition of prior learning are used. The supervisor verifies the authenticity and currency of the evidence and external assessors review the evidence for validity and sufficiency. This approach has an added advantage, as the learner is not required to fit in with some external

timetable, or attend some external assessment point.

Not  all ETQAs will  benefit  from  economies  of  scale.  Some  will  have primarily  small  Providers dispersed across the nation. Quality assurance systems will have to take this into account.

On the other hand, there are less likely to be economies of scale. Geography and the relatively small numbers of learners often means that it will not be cost-effective to pay monitoring and auditing visits to each Provider, each year. Auditing and monitoring is likely to place more emphasis on the submission of evaluations by all learners as well as on the reports issued by the Providers. A cyclical programme of visits to Providers is likely to be supplemented by visits to those where the reports submitted, or the evidence and evaluations submitted by learners, cause concern.

It will be desirable to assess underpinning knowledge and understanding especially where:

  • There is a larger knowledge base that underpins performance.
  • The outcomes of effective (or ineffective) performance are not easily attributable to the learner.
  • Solutions to problems have to be found where not all information is available.
  • The best solution today may not be the best solution tomorrow.

Ideally this would entail the use of problem-solving assignments and projects. An examinations body might undertake the setting and assessment of such assignments, if it has the necessary expertise. On the other hand, a sector may look to an education and training sub-

40                                                                                                        Quality Management Systems for ETQAs

6

Variations in the contexts in which the ETQAs Serving different sectors will operate that may impact  on their  QMS

system Provider to help meet this learning and assessment need, albeit perhaps at a distance. When an education and training sub-system Provider is used, an economic sector ETQA might co-operate with an education and training sub-system ETQA.

Providers  and  ETQAs  in  this  sector  may  form  partnerships   with  or  delegate  to,  colleague organisations in the education and training sub-system responsibilities relating to broader-based learning outcomes

Similarly, an economic sector Provider might look to an education and training sub-system Provider to meet the learning and assessment needs associated with purposes 2 and 3. The provision of basic skills development, for example, is often only economically feasible in larger businesses or in a location where a number of smaller businesses are willing and able to work together. To quality assure learning and assessment for these purposes, an economic sector ETQA might encourage economic Providers to form alliances with local Education and training sub-system Providers.

Alternatively an economic sector ETQA might form a partnership with, and in essence sub- contract the quality assurance to, an education and training sub-system ETQA. The effectiveness of such an arrangement would depend largely on the ability of the education and training sub- system ETQA body to concentrate on the desired outcomes, to value the diversity in context and not to prejudge the inputs and processes. If this proves to be too difficult, there may be potential for the economic sector ETQAs (especially the SETAs) to form a partnership with each other for

the purpose of quality assuring these broader learning outcomes.

ETQAs in this sector may reflect quality management systems in their industry or, in their absence, provide role models.

C: Economic sector ETQAs: Professional bodies

The following are in addition to the considerations identified above for SETAs.

ETQAs in this sector have wider, legislated responsibilities associated with protecting the public that will influence practice.

Professional bodies have wider responsibilities than just those pertaining to being an ETQA. Frequently these are delineated in Acts or Regulations. Normally, such bodies exist because the members of the public are not easily or immediately able to judge the quality of service provided by practitioners. For this reason, practitioners are required to demonstrate their competence before being able to practice. Given the specialised nature of these occupations, often it is only senior practitioners who are able to evaluate occupational competence and or issue a license to practice.

Quality Management Systems for ETQAs                                                                                                         41

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

In some ways, the existence of such bodies goes against the goals of the NQF. Specifically, they inhibit access to employment in particular occupations if their requirements have not been met. On the other hand, protection of the public is a higher good and so their role and responsibilities are warranted. But, there is often a suspicion that these responsibilities are used to control access and competition within the profession rather than for the protection of the public. Further, around the world, Professional bodies are being challenged to more effectively assure the continuing competence of practitioners.

Frequently the learning necessary to become competent,  and the evidence of it, entails multiple

Providers. It may also entail significant emphasis on self-directed  learning and assessment.

Frequently,  the  learning  required  to  become  an  effective  practitioner  in  a profession  is acquired through a variety of Providers. The nature of professional activities requires that individuals  have a sound  foundation  of knowledge  and well-developed  critical  thinking skills. This is necessary so that they can practice effectively under conditions where there is often uncertainty and ambiguity, where the problems frequently have unique characteristics and the best solution requires the careful balancing of alternatives each of which may have risks associated with it.

Most professional bodies depend on academic achievement to demonstrate these qualities. However, as the value of learning in other settings has increasingly been recognised and valued, professional bodies have begun to recognise ways in which this knowledge and these skills can be acquired and demonstrated outside of higher education. While the knowledge and skills developed in academia may be valuable in the practice of a profession, many professional bodies have concluded that, often, what is learned is insufficient for professional practice. Indeed, some of those who excel in academia lack fundamental skills such as the ability to communicate, empathize with and advise clients. Further, often the knowledge base developed in academia reflects a desire to push back the boundaries of understanding rather than instill that, which has to be used on a day-to-day basis in the practice of the profession. Increasingly, professional bodies are requiring potential applicants to have practicums or placements in order to develop these skills. There may be a requirement that this additional

knowledge and these additional skills be assessed during the practicum/ placement.

However, professional bodies often assess ownership of the practical knowledge base and intellectual skills through standardised assessments. Where there are critical technical and client interaction skills, these may be assessed through standardised skill application simulations.

Increasingly, however, professional bodies are concluding that the combination of evidence from these different sources is still inadequate. Many professional bodies have established, or are establishing an introductory grade of membership (for example, licentiateship) through which individuals can practice only under the supervision of a senior practitioner. Passage to full membership depends on the judgement of the senior practitioner. Steps are being taken to make assessment, and the auditing of assessment, by senior practitioners more valid and reliable.

The development and assessment of licentiates is in some way being aided by the growing concern  regarding  the  continuing  competence  of  practitioners.  There  is  a  trend  towards

Professional Bodies requiring practitioners to demonstrate continuing competence annually or

42                                                                                                        Quality Management Systems for ETQAs

6

Variations in the contexts in which the ETQAs Serving different sectors will operate that may impact  on their  QMS

at a frequency of between 2 to 5 years. Assuming a working life in a profession of about 30 years, in any year the number of people having to demonstrate continuing competence is between six and thirty times the number of new entrants. This is leading to the development of more effective strategies for assessing competence and effective performance in the work place and an increased emphasis on the individual’s responsibility for her or his own learning. These trends can only help enhance the effectiveness of initial assessment for new practitioners.

The development  of enhanced systems for assuring the continued  competence  of practitioners  is also enhancing the quality culture within professions.

However, the growing emphasis on assuring continued competence is unlikely to diminish the need for the professional body to assure the knowledge and skills base of the new entrant. A practitioner can cause much harm even when working under supervision. As ETQAs, the professional body, then, may need to assure the quality of learning and assessment in education, by the use of examination bodies that may be in-house or agents of the professional body, in the workplace and by the individual learner.

Providers vary from large educational institutions, through specialised testing bodies, through those who facilitate learning and assessment at a distance, to very small organisations and individual learners dispersed across the nation. As in the past, the most effective way of assuring quality in the education component is by ensuring that the standards meet the need, and to depend on the quality management systems that are put in place across education. A challenge is that the education programme may have a variety of purposes not limited to the preparation  of  professional  practitioners.  Progress  in  education  and  the  development  of research workers and content specialists are just three alternatives purposes. As a key stakeholder, membership of the relevant NSB/SGB would seem to be essential to ensure the relevance of the standards.

With  regard  to  the  assessment  of  practitioner  knowledge  and  skills,  the  technical soundness of examinations and tests are pertinent.

As to assessment in the work context, the emergence of the need to continually assure competence will provide new opportunities to better assure quality. It has been observed that one of the characteristics of a professional is that she or he continually monitors her or his performance and developments in knowledge, technology, systems and processes, identifies her or his learning needs and ensures that they are met. In defining the competence of practitioners, some bodies suggest that the ability to assess and coach colleagues is also a

requirement. Clearly, this would also facilitate learning and assessment in the work context.

Quality Management Systems for ETQAs                                                                                                         43

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

As an ETQA, these bodies have to consider that the practitioner  is often the facilitator  of learning and assessment. Quality management systems in this case tend to utilise self-assessment,  based on the techniques associated with the recognition of prior learning.

The practitioner is challenged to provide local verification wherever possible. Further verification and the collection of supplementary evidence are carried out through the oral testing of the practitioner (often over the phone). During each period, a specific proportion of practitioners is audited in rotation, plus those practitioners for whom there are doubts about the authenticity, currency or sufficiency of the evidence submitted.

ETQAs in this group are charged by legislation to define competence and what constitutes  a quality service. They are required to ensure a quality culture within the profession.

Summary

ETQAs in the two sectors, education and training sub-system and economic sector have to deal with Providers of different size and culture who have different, wider, purposes. While the fundamentals of a quality management system for ETQAs are relevant to each, the emphasis within the systems operated by the different sectors and the mechanisms adopted will vary. The outcomes, however, should be the same; the development of a quality culture that benefits

learners and society as described by the objectives of the NQF.

Table  5: Some Differences between ETQAs from the two  identified Sectors that  will  influence their Quality

Management Systems

Economic Sector

Education and Training Sub- System Sector ETQAs will frequently:

Sector Education and Training Authorities (SETAs) & Professional, Regulating and Licensing ETQAs will frequently:

Professional, Regulating and Licensing ETQAs additionally will frequently:

Often Providers are significantly larger and have many more learners for qualifications than will be the case for ETQAs in the economic sector.

Tend to work with smaller providers and smaller numbers of learners than will be the case for education and training sub- system ETQAs. But some may have relatively large providers.

Work with multiple Providers and look for significant emphasis on self-directed learning and assessment.

In the main, assure learning and qualifications associated with broader-based outcomes

Assure learning and qualifications associated with an industrial or commercial sector and work performance

Assure learning and qualifications and standards associated with a single or closely related occupation(s).

44                                                                                                        Quality Management Systems for ETQAs

6

Variations in the contexts in which the ETQAs Serving different sectors will operate that may impact  on their  QMS

Table 5 continued

Economic Sector

Education and Training Sub- System Sector ETQAs will frequently:

Sector Education and Training Authorities (SETAs) & Professional, Regulating and Licensing ETQAs will frequently:

Professional, Regulating and Licensing ETQAs additionally will frequently:

Need to delegate responsibilities to Providers to be cost effective. Providers, in turn, may delegate responsibilities within the organisation. Responsibilities may also be delegated to examinations bodies for the testing of basic knowledge and skills where required, or for quality assuring Providers’ assessments of basic skills and knowledge. But Provider and ETQA remain accountable

Where Providers are associated with employment sectors that are seeking to establish quality management systems, will be unlikely to experience a great deal of difficulty in doing likewise. However, the ETQA will want to ensure compatibility with the requirements of the NFQ system.

Not likely to benefit from economies of scale. Some will have primarily small Providers dispersed across the nation. Quality assurance systems will have to take this into account.

Have to take into account that, the practitioner is often the facilitator of learning and assessment. Quality management systems in this case will tend to utilise self- assessment based on the techniques associated with the recognition of prior learning.

Need to phase in initial accreditation requirements regarding learning and assessment facilitator competence over an extended period of time.

Need to take into account

that much of the learning and assessment will take place in the workplace in this sector.

Have wider, legislated responsibilities, associated with protecting the public that will influence practice.

Be relatively large, have specialist expertise in learning and assessment as well as quality assurance and a formal internal quality management system led by a designated quality manager.

Be relatively small, have sector specialist as well as quality assurance expertise, and an internal quality management system that reflects easier internal communication.

Form partnerships with or delegate to, colleague organisations in the education and training sub-system, quality assurance relating to broader- based learning outcomes.

Find it easier to estimate the added-vale of quality management systems and therefore assure and demonstrate their cost- effectiveness.

Benefit from, and need to take into account that enhanced systems for assuring the continued competence of practitioners will also enhance the quality culture within professions.

Model quality management systems for their Providers.

Where QMS is already embedded in sector, reflect quality management systems in their industry. Others will provide a lead

Be charged by legislation to define competence and what constitutes a quality service. They are required to ensure a quality culture within the profession.

Quality Management Systems for ETQAs                                                                                                         45

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

Appendix 1

The Purposes for Learning

The most important characteristics of quality are usually:

  • relevance (of what is learned to the purpose of learning);
  • the learning process (the process enabling the individual to learn);
  • accessibility (having in mind the learner’s circumstances); and
  • currency (having regard for the purpose for learning).

For qualifications and standards, in addition to relevance and accessibility, reliability and currency are the important characteristics of quality.

Relevance, or the purpose for learning, pertains to client needs and expectations. A product or service that does not meet these needs or expectations is not relevant; it is of inferior quality. So assuring relevance is an important step in assuring quality. In the context of education and training, there are two important questions that have to be addressed regarding clients’ needs and expectations:

  • Why is the learner willing to invest time (and possibly money) in a particular education or training activity?
  • Why is the sponsor(s) willing to make the investment in the learners’ learning?

The answers to these questions define or inform the definition of ‘relevance’. There are many reasons for learning. Table 6 lists the primary ones.

Table  6: Primary Reasons  for  Learning

1      Seek and find fulfillment in life.

2      Contribute  to  the stability  and growth  of society  and the well being  of all members  of a community.

3      Progress in the field of education.

4      Respond effectively to future situations and expectations.

5      Meet his/her own economic needs and to contribute  to the economic survival and growth of a community  throughout  her/his working life.

6      Gain employment in specific sectors of employment.

7      Gain employment in specific jobs/occupations.

8      To be recognized as competent  in an occupation  or job.

9      Improve performance in specific work roles or tasks.

10    Learn new skills and knowledge applicable to specific work roles or tasks.

In the context of the NQF, the NSB regulations state that the SGB defines the learning outcomes, but the learner should be able to select the packages of learning outcomes that serve her or his purpose for learning, perhaps with the advice of the Providers.

46                                                                                                        Quality Management Systems for ETQAs

Definition of terms

Assessment       means the process of collecting evidence of learners’ work to measure and make judgements about the achievement or non-achievement of specified National Qualifications Framework standards and/or qualifications

Assessor            means the person who is registered by the relevant Education and Training Quality Assurance Body in accordance with the criteria established for this purpose by a Standard Generating Body to measure the achievement of specified National Qualifications Framework standards and/or qualifications, and “constituent assessor” has a corresponding meaning

Audit                   means the process undertaken to measure the quality of products or services that have already been made or delivered

Education and Training Quality Assurance Bodies means a body in terms of section

5(1)(a)(ii) of the SAQA Act, responsible  for monitoring  and auditing achievements  in terms  of national  standards  or qualifications,  and to which  specific  functions  relating  to  the  monitoring  and  auditing  of national standards and/or qualifications have been assigned in terms of section 5(1)(b)(i) of the SAQA Act

External customer/s means an external person, body or structure that is a beneficiary to the services or product of a particular organisation

Facilitator of learning and assessment means an individual who facilitates learning processes and activities and manages and administers assessment, educator, teacher, trainer, mentor etc

Internal customer/s  means an internal person, body or structure that is a beneficiary of the services and/or products of a particular organisation

Moderator           means a person, body or organisation that ensures that the assessment of the outcomes described in National Qualifications Framework standards and/or qualifications is fair, valid and reliable

Moderation         means the process which ensures that assessment of the outcomes described in the National Qualifications Framework standards and/or qualifications is fair, valid and reliable and “Verification” has a corresponding meaning

Moderating Body  means a body specifically appointed by the Authority for the purpose of moderation, across ETQAs

National Standard Body   means a body registered in terms of section 5(1)(a)(ii) of the SAQA Act responsible for establishing education and training standards or qualifications, and to which specific functions relating to the registration of national standards and/or qualifications have been assigned in terms of section 5(1)(b)(I) of the SAQA Act

Professional Body  means a statutory body registered as such in terms of the legislation applicable to such bodies, or voluntary body performing the functions contemplated in the legislation for such bodies but not registered as such

Provider              means a body which delivers learning programmes  which culminate in specified   National   Qualifications   Framework   standards   and   /   or

qualifications, and manages the assessment thereof

Quality Management Systems for ETQAs                                                                                                         47

DIRECTORATE: QUALITY ASSURANCE AND DEVELOPMENT

Service Contract  means contractual working agreement, between an ETQA and its customers and/or agencies

Standard Generating Body    means a body registered in terms of section 5(1)(a)(ii) of the SAQA Act, responsible for establishing education and training standards or qualifications, and to which specific functions relating to the establishing of national standards and/or qualifications have been assigned in terms of section

5(1)(b)(i) of SAQA the Act

Bibliography

  1. Huba M.E. & Freed J.E. (2000), Learner-Centered Assessment on College Campuses. Needham

Heights, MA, Allyn and Bacon

  1. Swanson D.B, Norman G.R. & Linn R.L. (1995), Performance-based Assessment: Lesson from the

Health and Professions, Educational Researcher, 24 (5), 5-11

  1. Simosko S. & Cook C. (1996), Applying APL principles in Flexible Assessment, (pages 99-103) London, Kogan Page
  1. Schimberg B. (1980), Occupational Licensing: A Public Perspective, Princeton USA, Center for

Occupational and Professional Assessment, Educational Testing Services

  1. Kantner R. (1994), The ISO 9000 Answer Book, (page 11), Essex Junction, VT, Oliver Wright

Publications

  1. Foster A. (1990), Explaining Quality Systems, London, Training Tomorrow (page 38), MBC University

Press Ltd

  1. Malcolm Baldrige National Quality Award c/o American Society for Quality, P.O. Box 3005, Milwaukee, WI 53201-3005
  1. Waller J, Allen D. & Burns A. (1993), The Quality Management Manual, How to Write and Develop a

Successful Manual for Quality Management Systems” (page 141), London, UK, Kogan Page Ltd

  1. Walkin L. (1992), Putting Quality into Practice, Cheltenham, England, Stanley Thornes (Publishers) Ltd

48                                                                                                        Quality Management Systems for ETQAs

Funded by the European Union under the European

Programme for Reconstruction and Development

South African Qualifications Authority

Postnet Suite 248, Private Bag X06, Waterkloof, Pretoria, 0145

Tel: 012-482 0800  Fax: 012-346 5812/3