Quality assurance in higher education in Easter Europe,...

QUALITYASSURANCEINHIGHEREDUCATIONINEASTERNEUROPE,CAUCASUS,CENTRALASIA

ANDMONGOLIA

TheStateofAffairsandtheImpactofQualityAssuranceandAccreditationonTertiaryEducationinEasternEurope,Caucasus,CentralAsia,andMongolia

SCOPINGSTUDY

REPORT

March2014

s_cochet

Typewritten Text

ED/2014/PLS/HED/04

s_cochet

Typewritten Text

ReportonaScopingStudyofQualityAssuranceinHigherEducation

2

FOREWORDThe International Network of Quality Assurance Agencies in Higher Education(INQAAHE) has undertaken the current scoping study under the auspices ofUNESCO.INQAAHE had already in the past been cooperating with UNESCO in theframework of the so‐called Global Initiative for Quality Assurance Capacity(GIQAC) project. In continuation of this collaboration, INQAAHEhas nowbeencommissioned to execute a Scoping Study to better understand the qualitydevelopmentsinthecountriesofEasternEuropetheCaucasusaswellasCentralAsia (including Mongolia). More specifically, the study aims at identifyingdevelopmentaltrendsandgoodpracticesinthesethreeregionswhileprovidingrecommendationsforfutureimprovementandenhancement.TheScopingStudywas launchedonNovember29th,2013andendedonMarch31st,2014.TheINQAAHEBoardsetupateamofexpertstoconducttheScopingStudy.Theteamcompositionisasfollows:

‐ Teamleader:SusannaKarakhanyan,PhD,INQAAHEVice‐President‐ IringWasser,INQAAHEBoardMember,CEENQAPresident‐ GaneshHegde,INQAAHEBoardMember‐ MilaZarkh,CEENQASecretariat

TheFinalReportwasdiscussedattheINQAAHEBoardleveltobringindiversityofinsightsofalltheBoardmembers–representingallthemajorregionsoftheworld–aswellastovalidatetherecommendations.ThisreportconstitutestheFinalReportandisinlinewiththeTermsofReferencetoINQAAHE.Theresearchteamextendsits gratitudetoallthose,whohavecontributedtothe

productionofthisreport:theexternalqualityassuranceagenciesintheEasternEurope,Caucasus,Central

AsiaandMongolia.

SpecialthanksandappreciationgotoDavidWoodhouseandLeeHarvey

whohavekindlycontributedtothereportproductionwiththeirinsightfulfeedback.


3

TableofContents

EXECUTIVESUMMARY....................................................................................................4

I. INTRODUCTION..........................................................................................................9

II. GENERALANDSPECIFICOBJECTIVESOFTHESTUDY.................................11

III.METHODOLOGY......................................................................................................12

IV.MAJORDEVELOPMENTSINQUALITYASSURANCEINEASTERNEUROPE,

CAUCASUS,ANDCENTRALASIA................................................................................13

1.GeneraloverviewofQAsystems...................................................................................13

2. QualificationsFrameworkandLearningOutcomes...........................................15

3. TheQualityAssuranceProvider,GovernanceFrameworkandMission......16

4. HumanandFinancialCapacity...................................................................................17

5. ExternalEvaluationProcedures................................................................................18

6. QualityAssessmentCriteria:developmentandimprovement.......................23

7. Accountabilitymechanisms........................................................................................24

8. StakeholderInvolvement............................................................................................25

9. CapacityBuilding............................................................................................................26

10. Networking....................................................................................................................27

V. CONCLUSIONS..........................................................................................................28

VI.APPENDICES.............................................................................................................30

AppendixA:ListofAcronyms.............................................................................................30

AppendixB:ListofParticipantCountriesandAgencies............................................31

AppendixC:ContactdetailsofParticipatingQAAgencies........................................33


4

EXECUTIVESUMMARY

he International Network for Quality Assurance Agencies in HigherEducation (INQAAHE) in line with its mandate of being an umbrellaorganization of all the quality assurance (QA) agencies and networks at

international level has been actively promoting the establishment andenhancement of quality culture in its member countries including EasternEurope, Caucasus, Central Asia and Mongolia. To better understand thedevelopmentaltrendsandtoidentifygapsaswellasgoodpractices,theScopingStudyontheStateofAffairsandtheImpactofQualityAssuranceandAccreditationon Tertiary Education in Eastern Europe, Caucasus, Central Asia andMongoliaundertheauspicesofUNESCOwasinitiated.Theobjectivesofthecurrentstudywereasfollows:

‐ To assess the situation of quality assurance organizations in Eastern Europe(South Eastern Europe), Caucasus, and Central Asia. It targets both countrieswith established QA entities aswell as countrieswith no such entities. In theabsenceofaformalentityforQA,thestudytargetedQAactivitiesinthecountry;

‐ To identify gaps in areas in the deployment of QA systems in the targetcountries;

‐ ToassesstheimpactofQAinthesub‐regionswitha focusonthe landscapeofQAintheregion(s).

‐ Toproviderecommendationsforfutureimprovementandenhancement.

As a result of the scoping study the followingmajor findings, subdivided inareas of good practice, areas for future improvement and relatedrecommendationshavebeenidentified:

EXTERNALREFERENCEPOINTS

GoodPractice

‐ Establishment of National Qualification Frameworks in most countries underreview;

AreasofConcern

‐ InmanyinstancesNationalQualificationsFrameworks(NQFs)areintheprocessofestablishmentand,insomesystems,toolsmeasuringandassessingalignmentofstudy/academicprogrammeswiththeseNQFshavebeenlaunched.However,mostofthesystemshavenotstartedtheprocessofalignmentyet;

‐ The external quality assurance frameworks of some of the countries areabundantwithmultipleQAtools—institutionalregistrationandaccreditation,programme evaluation and accreditation, attestation, monitoring supervision

T


5

etc.Frequentlytherearenoprovisionsinplacetousesynergiesbetweenthesevarious forms of QA, duplicating work and increasing the burden on highereducationinstitutions(HEIs).

Recommendations‐ To ensure a coherent, complementary and consistent approach to external

qualityassurancethecountriesinthesampleshouldreviseandcoordinatethevarious external quality assurance mechanisms in place at the national level.Such a revision should aim at enhancement of HEI´s educational performancewhilealleviatingtheiradministrativeburden;

‐ Toestablishandoperationalizetoolsensuringalignmentofthestudy/academicprogrammeswiththeNQFsandcheckingsuchanalignment.

GOVERNANCEANDINDEPENDENCE

GoodPractice

‐ InclusionofinternationalexpertsintheGoverningBoardsoftheexternalqualityassuranceagencies(EQAAs),whichcontributes todiversityand impartiality indecision‐takingtoaconsiderableextentwhileatthesametimereducingconflictofinterests;

AreasofConcern

‐ Governments andministries’ interference in the external quality assurance ofHEIsandprogrammesinmanycountriesisstilltangible,oftentothedetrimentofHEI’sautonomy,accountabilityand,therefore,qualityofprovisions;

‐ Governanceoftheagenciesisstillinfluencedbytheparticipationofministryandgovernmentrepresentatives,who,insomeofthecasespresideinthedecision‐makingbodiesonanexofficiostatus.Ontheotherhand,someofthegoverningbodies include internationalexperts,which, tosomeextent,brings indiversityandimpartialityindecision‐taking;

‐ Independenceofdecision‐takingisnotensuredinmostofthecases;

Recommendations

‐ Reconsider the role of the ministries and governments by limiting theirinvolvement in independentexternal reviewtoallow for independentexternalqualityreviewanddecision‐making.

STAKEHOLDERINCLUSION

GoodPractice

‐ Stakeholder inclusioninvariousqualityassuranceactivitieshasbeenobservedasamajortrend.


6

AreasofConcern‐ Stakeholder involvement in quality assurance is in place to some extent. Such

major impediments as restrictions in the legal framework, lack of funding,scarcityofcapacitybuildingeventsanddisseminationactivitiesandtoalesserextent lack of inclusion in the decision‐making processes have however beenidentified; increased involvement of the EQAAs in the international networksseemstobeatrend.However,apro‐activeparticipationisstilltobedesired.

Recommendations‐ Devise and implement consistently policies and approaches in regard to

stakeholder involvement inexternalqualityassurancebyconveyingthevaluesandbenefitsoftheexercisetothestakeholders.

USEOFINTERNATIONALEXPERTS

GoodPractice

‐ Theexternalreviewers’panelsincludethekeystakeholders—facultymembers,students,andemployers—inmostofthecases;

‐ In some cases, there is representation of international peers on the externalreviewers’panelsandconductoftheproceduresintwolanguagesforincreasedrecognitionattheinternationallevel

EXTERNALQUALITYASSURANCESTANDARDSANDPROCEDURES

GoodPractice

‐ The external evaluation procedures follow the generally accepted stepsrevolving around self‐evaluation, desk‐review, site‐visit, panel reportproduction,decision‐takingbyrespectiveaccreditationcouncilsorcommissionsandfollow‐upconductedbytheHEIsandoverseenbytheEQAAs;

‐ Development of criteria in most of the cases is done by benchmarking theinternational good practice, identifying local needs and refining the criteria tolocal needs. Such criteria are ideally piloted prior to full implementation toensurelegitimacyoftheadoptedapproachesandrelevancetothecontext;

AreasofConcern‐ In someof thecasesadirect transferof criteriawithout further refinement to

thelocalneedshasoccurredleadingtothedamagetothesystem;

Recommendations‐ Whilerevisingthecriteria/standardsforexternalevaluationthefollowingsteps

shouldbeundertakentoensurethecriteriadomeettheneedsofthesystemandlead to its enhancement: (1) benchmarking with international trends; (2)identificationof localneeds;(3)developmentof thedraftcriteria/standards inbroad consultation with the stakeholders; (4) pilot of the criteria/standardspriortofullimplementation;(5)refinementandofficialadoption.


7

ACCOUNTABILITY

GoodPractice

‐ Within the EQAAs development and operationalization of such accountabilitymechanisms,asinternalqualityassurancemechanismhasbeenobservedandiscommendable.

‐ Some well‐established agencies have undergone external recognition againstinternationally accepted standards (ENQA ESGs), which contributes to theirinternationalstandingandcredibility.

‐ IncreasedinvolvementoftheEQAAsintheinternationalnetworksseemstobeatrend.

AreasofConcern‐ Whilesomeoftheagencieshaveacertainnumberofaccountabilitymechanisms

inplace,mostofthemarestill intheprocessof introducingandrefiningthem.Development andoperationalization of internal quality assurancemechanismsat the EQAA level seems to be not adequately institutionalized inmost cases.Furthermore, such exercise as external recognition of the agency againstinternationally accepted standards (ESGs,GGP, ChibaPrinciples) is atminimallevelinsomesystemsunderreview.

Recommendations

‐ Develop and implement a robust internal quality assurance system withinEQAAs to ensure continuous improvement of the services as well asaccountability.

‐ MakeusebyEQAAsofinternationallyacceptedstandards(e.g.ESGs,GGP,ChibaPrinciples) to enable international recognition of EQAAs against thosestandards.

FINANCIALANDHUMANRESOURCECAPACITY

AreasofConcern

‐ Capacity building of the EQAA staffmembers is essential to the successof theexternal evaluation endeavour. Also the role of the staff varies considerably,someusingthemmainlyforadministrativepurposes,somegivingthemamoreprominent/responsibleroleintheQAendeavours.Mostofthesystemshavealsoinadequatestaffingtoensuresystem‐wideresearch,identificationoftrendsandfurtherenhancementofthesystem;

‐ Amajorneedincapacitybuildingofexternalevaluatorshasbeenrevealed.Suchneedsmainlyevolvearoundinterpretationofthecriteria/standards,evaluationbased on the criteria/standards, production of the final report by the panel,development of recommendations that are clear enough and valid as well asclearapproachestofollow‐uptopromotesystemenhancement;


8

‐ Financial capacity of the EQAAs in the target countries is at risk because ofunderfundingfromthegovernmentandlackofalternativerevenuesources.

Recommendations‐ Engage in major capacity building activities for external reviewers to ensure

adequateinterpretationofthecriteria/standards,necessaryskillsinevaluationbased on the criteria/standards, production of the final report by the panel,developmentofrecommendationsthatareclearenoughandvalid.

‐ EnsurecontinuousprofessionaldevelopmentandcapacitybuildingfortheEQAAstaff to enable them to participate more actively in the external qualityassurance reviews and to perform such major functions as system‐wideresearch, identification of gaps and developmental trends and creation ofrecommendationsforimprovementandenhancement.

‐ Revise the approaches to funding mechanisms to enable cash flow fromalternativesourcessuchasconsultancyandcapacitybuildingservicesaswellasimplementation of international and local grants to be less dependent on thegovernmentalallocations. Itgoeswithoutsayingthatdiversificationof incomesourcesmustnot infringeon thequality and independenceof the externalQAdecision‐makingprocess.

‐ Ensurepro‐activeparticipationofEQAAsandHEIsintheinternationalnetworksandtakeadvantageofinvolvementindevelopmentalactivitiescarriedoutbythenetworks.

‐ ***‐


9

I. INTRODUCTION

ntandemwithglobalizationandmassificationthehighereducationsystemsin EasternEurope, Caucasus and Central Asia, like all other systems in theworld, experienced mushrooming of a diversity of HE providers, some of

them with a questionable quality. As a result, perspectives on the quality ofprovisionsandrelevanceoftheawardedqualificationstothemarketneedswereat risk. Thus, the countries in Eastern Europe, Caucasus and Central Asiaembarkedona journeyof rethinking thearchitectureof their systems toalignwith international trends, therebyenhancing thequalityandrelevanceof theirprovisionsaswellasbeingvisibleintheinternationalarena.While some countries (e.g. USA) have enjoyed a history of over a century inexercising quality assurance in HEIs in a systemicway, the cases in the threesub‐regionsarenewlyemergingones.UnlikeindecentralizedsystemswheretheinitiativeforexternalqualityassuranceingeneralandaccreditationinparticularcamefromtheHEI leaders(e.g. theUSAandNewZealand)the initiativeof theestablishmentofQAsystemsinthetargetregionspredominantlycamefromtherespectivegovernments intheirsearchforbettermechanismstosafeguardthequalityofHE.Being concernedabout thenature andperformanceofHEIs andtheir impact on the economic development, respective governments, in theirsearchtoestablishmeasuresforaccountability,tookdecisionstoestablishsuchexternalmechanisms as licensing and accreditation. In some systems it took aformofaquasi‐autonomousQAagency, inothers thegovernmentsestablishedunitswithin respectiveministries to take upon themission of external qualityassuranceinanefforttokeepcontrol.One of the changes that most affected the higher education sphere was theBolognaDeclaration in1999withagoal to increaseacademicandprofessionalmobilityonthebasisofrecognizedandtransferablequalifications.Thelaunchofthe Bologna process became also an impetus to a revised approach to qualityassurancebyemphasizinganindependentandenhancement‐drivennatureofitby many of the countries in the three regions. Educational policymakers inEastern Europe, Caucasus and Central Asia, along with their Europeancounterparts,identifiedtoolsthatcouldbeusedtopromotethecommongoalsofrecognitionandpromotionofprofessionalandstudentmobility.Thus,theneedto establish independent and robust external and internal quality assurancemechanisms to attest to the public about the validity of theHE Providers andcredentials was tangible. Consequently, most of the countries in the areaembarkedon establishment of quality assurancemechanisms, both at nationalandinstitutionallevelsandbroadlyexercisethem.The International Network of Quality Assurance Agencies in Higher Education(INQAAHE)inlinewithitsmandateofbeinganumbrellaorganizationofalltheQA agencies and networks at international level has been actively promotingestablishment and enhancement of quality culture in its member countries

I


10

including the 3 sub‐regions. Thus, to better understand the developmentaltrends,identifythegapsandgoodpractices,thisScopingStudywasinitiated.Asa result of this study a set of recommendations is addressed to furtherimprovementandenhancementofthetargetsystems.


11

II. GENERALANDSPECIFICOBJECTIVESOFTHESTUDYTheobjectivesof theScopingStudyontheQualityAssurancestateofaffairs intheEasternEurope,CaucasusandCentralAsiaareasfollows:

1. AssessthesituationofqualityassuranceorganizationsinEasternEurope(South

Eastern Europe), Caucasus, and Central Asia. It targets both countries with

establishedQAentitiesaswellascountrieswithnosuchentities.Intheabsence

ofaformalentityforQA,thestudytargetedQAactivitiesinthecountry;

2. IdentifygapsinareasinthedeploymentofQAsystemsinthetargetcountries;

3. AssesstheimpactofQAinthesub‐regionswithafocusonthelandscapeofQAin

theregion(s).

4. Providerecommendationsforfutureimprovementandenhancement.

Theexpectedoutcomesevolvearound:

1. BaselineinformationthatwillgiveabetterunderstandingoftheQAsituationin

theEasternEurope,CaucasusandCentralAsiaincludingMongolia;

2. Identifiedissues,challengesandgapsinQAintheEasternEurope,Caucasusand

CentralAsia;

3. IdentifiedQAgoodpracticesthatshouldcontributetotheimprovementofaQA

cultureintheEasternEurope,Caucasus,CentralAsiaandMongolia.

4. Recommendationson furtherenhancementofqualityassurance in theEastern

Europe,Caucasus,CentralAsiaandMongolia.


12

III. METHODOLOGYThemajormethodsforundertakingtheScopingstudyevolvedaround:

‐ AsurveyaddressedtothebodiesthatarecarryingoutfunctionsofqualityassuranceinthecountriesofEasternEurope,CaucasusandCentralAsia(Mongoliaisalsoincludedinthesample).Thequestionnairedesignedforthisstudyintendedtolookatsuchmajordevelopmentsinthesphereofquality assurance asnatureof theQAprovider, governance frameworkand mission, human and financial capacity, the external evaluationprocedures undertaken by the bodies, the QA criteria and theirdevelopment and improvement nature, accountability mechanisms,stakeholder involvement, capacity building events undertaken by therespective bodies to enhance the system as well as the networkingcapacityofthosebodies.

‐ Analysisofthedataobtainedfromwebsitesoftherespectivebodiesandinternationalnetworks.

The survey was distributed to 33 bodies charged with quality assurancefunctionsin28countries(seeAppendixBandCforthelistofthecountriesandparticipatingagencieswithcontactdetails).The questionnaire was distributed electronically via eSurvey athttps://www.esurveycreator.com. Both the paper based questionnaire and thelink to theonlinesurveyweresent to34agencies/ministries in29countries inEasternEurope,CaucasusandCentralAsiaincludingMongolia.Wehavereceived23(67.64%) responses from 19 (65.5%) countries. Almost all the qualityassurance agencies responded. The countries, where the QA function is stillperformedbytheministriesofeducation(27.5%)didnotrespondtothequery.Alanguagebarriermighthavebeenamajorfactorforthenon‐response.

https://www.esurveycreator.com


13

IV. MAJORDEVELOPMENTSINQUALITYASSURANCEINEASTERNEUROPE,CAUCASUS,ANDCENTRALASIA

1.GeneraloverviewofQAsystemsIfwehaveanoveralllookatthethreeregions,outof29countriesinthetarget8(27.5%)didnothaveQAagencies and theministriesof education and scienceperformed the quality assurance functions. Below is the distribution of QAfunctionsperregionandjurisdiction:

Chart1:DistributionofQAfunctionsperregionandjurisdictionAs seen from Chart 1, in Central Asia (in spite of the existence of the CentralQualityAssurancenetworksandongoingactivitiestocreatenewQAentities,e.g.intheframeworkofthecurrentQUEECAproject)andtosomeextentinEasternEuropeandCaucasus,ministriesarestilltryingtokeepcontroloverthesystem,thus implementingthequalityassurancefunctions. Inthesesystems,accordingtotherespondents,independentEQAcontinuestobecompromised.ContinuousreformofthesesystemsisneedediftheEQAistobeoperationalizedsuccessfullythus leading to system enhancement, relevance and, therefore, internationalvisibility in the long run.However, the legacy of a centrally controlled systemshould be kept inmind, while revising the approaches to governance, since atotallyindependentandprivately‐runmodelofEQAAseemstobelessfeasibleinsuch contexts. The primary reason for such a concern is being under thejurisdiction and protection of the respectiveministries and already entitled toconferdegreesthepublicHEIsareactuallyreluctanttoundergoproceduresthatarenotstipulatedbythegovernment.As for the age and extent of maturity of the EQAAs, establishment of suchagencies is at various levels in the target countries. Some havemore than 20years of experience being established in the early 1990s, others are newlyemerging ones established in 2011. However, with regards to maturity some

Caucasus EasternEurope CentralAsia+Mongolia

3

18

215 5

QAA Ministry


14

systems, albeit the history of over 15 years, are still being challenged andexplicitlystateaneedforamajorrevisionoftheapproaches.Most agencies in the scopewere theonly agencies at the country level todealwithqualityassurance.In7countriesoutof19thatresponded,however,morethanoneagencywasoperating–bothatthesubjectspecificandgenerallevels.91.3%of the respondentsareoperating ina single countryandonly13%areoperatinginternationallyaswell.Further,inthecountriesthatrespondedtothequestionnaire there are 61 quality assurance agencies, out of which 28 aresubjectspecificand33generallyoperatingagencies.Alltheagenciesinthesamplecoversuchtypesofprovidersaspublicandprivateuniversities. In addition to the public and private HEIs, 13 agencies (56.5%)covernon‐universityprivateandpublicprovidersand4agencies(17.4%)covertransnationalprovidersaswell.Further, almost all the respondents are involved in one ormore internationaland regional quality assurance networks. The distribution of networkmembershipsisasreflectedbelow:

Chart2:Membership/affiliationtonetworks1AsseenfromChart2,thepopularityofsuchagenciesasCEENQA,INQAAHE,andENQAisstriking.APQNisalsopopularinthethreeregionstosomeextent.Furthermore,ENQA(ESGs),INQAAHE(GGP),APQN(Chibaprinciples)alsohaveestablished standards against which they conduct external evaluation of theagencies for recognition purposes and ENQA also does it for granting amembershipstatus.Amongtheagenciesthatrespondedtothesurvey,7agenciesareENQAfullmembersoutofwhichthetermforoneoftheagenciesexpiredin2012. The rest are ENQA associates, which means they are still supposed toundergo external evaluation against ESGs to become full members. As for theINQAAHE GGP alignment or external recognition against the APQN Chiba

1SeeAppendixAforacronyms

0 5 10 15 20

INQAAHE

ENQAmember/associate

CEENQA

APQN

ECA

EAQAN

CHEA‐CIQG

EQANIE

Others


15

principles, none of the respondents thus far has used this external referencesystem and only some planned to do so in the near future (see Section 7:Accountabilitymechanisms).

2. QualificationsFrameworkandLearningOutcomesEstablishment of a National Qualifications Framework (NQFs) in the threeregionsplusMongoliaisatvaryinglevels.TheownershipofitsestablishmentispredominantlywiththeMinistryofEducationandSciencewith involvementofsuch stakeholders as HEIs, employers and students. In few exceptional casescountriesestablishedNQFCouncilswithinclusionofkeystakeholderstodeveloptheNQFs.Thus,theNQFsareinplacein77.3%(17countries)andendeavorsaretakentoestablishoneintherest(22.7%,5countries).However,inthecountrieswhereitdoesexistitisstillatitsinitialphaseandrobustqualityassurancemechanismsarestilltobeestablishedtoensureitsimplementation.Againstthisbackground,noneoftheNQFshasactuallybeenself‐certified–aprocedureundertakenwithinclusion of international experts to certify NQF implementation forinternational recognition purposes. In some countries pilot self‐certificationproceduresrevealedmajorgapsbetweentheprogrammeobjectivesandNQFs.AsfortheSectorialQualificationsFrameworks,45.5%oftherespondentsstatedhavingnoneand54.5%statedthatthecountryhasestablishedSQFsinsomeofthesectorsoronlyforthesectorsregulatedbythegovernment.The open‐ended questions delved deeper into the processes of developingNQFs/SQFsandstakeholderinvolvementintheprocessaswellasthemodalitiesofassessingachievementoflearningoutcomes.Themajorreflectionsregardingthedevelopmentprocesswereasfollows:

‐ establishment of committees per subject area including such stakeholders asHEIs,practitioners,ministries

‐ the QAA agency develops a proposal by referencing SQF levels to EQF andsubmitsittothegovernmentforapprovalandlegalization.

However,formostofthecountriesthedevelopmentsarestillintheprocessandtheprocesswasnotdescribed.AsfortheQAcriteriaensuringalignmentandassessmentofthealignmentoftheprogrammes with the qualifications frameworks, 76.2% of the respondentsstated they included respective points in their criteria. Thus, the agenciesactually assess the achievement of the learning outcomeswithin the academicprogrammesdefined in theNQFs/SQFs, through including it in thecriteriaandthecriterialookattheachievementthrough:

‐ evaluationofgraduatepapersandmasterthesis;‐ application of relevant questionnaires and recommendations for the experts

undertheauspicesoftheMinistry;‐ checkingtheprogrammesagainsttheitemsintheNQFs.


16

3. TheQualityAssuranceProvider,GovernanceFrameworkandMissionMostofthegovernmentsinthethreeregionshaveadoptedrespectivelegislationto introduce external quality assurance systems. Some countries opted forestablishment of independent agencies (34.8%), 56.5% of the agencies in thesample are quasi‐autonomous national agencies, 8.7% are supranational non‐profitagenciesunder the jurisdictionof theMinistryofEducationandScience.Veryfewactasanadvisorybodytothegovernment.MostoftheagencieshaveAccreditationCommissions/Councils,whichalsoactasGoverning Boards charged with overall governance of the agency’s activities.Only invery fewcases thereare twomajordecision‐makingbodies–BoardofTrusteesresponsibleforstrategicgovernanceandtheAccreditationCommissionchargedwithamissiontotakedecisionsonaccreditation.As for the representation of the stakeholders in the decision‐taking bodies asignificantvariationinthecompositionofthegoverningbodieswasregistered.In 43.5% of the cases it includes representative of the government/ministry,tertiary education institutions (public and private) (78.3%), professionalassociations (52.2%), students (87.0%), industry/business representatives(56.5%)andinternationalexpertisrepresentedin6cases(26.1%).Withregardstothemissionoftheagenciesinthesample,theagenciesconsiderthe followingareas as of high importance to the agency: quality enhancement,accreditation,transparency,accountability,studentprotectionandinternationalcomparability.Asforthefunctions,86.9%statedconductinginstitutionaland91%programmeaccreditation. Additionally, some organizations stated conducting institutionalrecognition (13%, 3 cases), programme evaluation (47.8%) and 17.3%stateddealingwithcross‐borderprovidersasoneoftheirfunctions.

Chart3:FunctionsperformedbytheEQAAsTheoverallconcernofsomeoftheagenciesisthattheywereobligedtoconductmore than three types of procedures, which overburdens them and, actually,does not allow for a better performance at the individual procedure levels.


17

Undoubtedly,provisionofmultipleprocedurescausesbothfinancialandhumanresource constraints for some of the agencies while at the same time alsoincreasing the administrative burden for the HEIs in various countries underreview.Asforthenatureoftheprocedures,theyarepredominantlyvoluntaryinnatureifdonebytheagenciesnotaffiliatedtotheministries.Thisisespeciallythecaseat programme level. The point might be made, that this approach is moresuitable for mature QA systems and established quality cultures. In newlydeveloping systemswith a centrally controlled legacy, amandatory nature, atleast for the first twocycles,ofprocedureswith links to fundingwouldensuresystem‐wide developments and enhancement if devised specifically to fit theneeds of the system (inmost of the systems borrowing of criteria frommoremature systems without adjusting to the local needs has had seriousconsequencesonthequalityofprovisions).Thus, a major implication is most systems are still in development and it isextremely important to set clear and robust quality assurance mechanismsleadingtoenhancementratherthanactingasburdenfortheHEIs.

4. HumanandFinancialCapacityWithregardstothecapacitytoundertakethequalityassuranceprocedures,theagencies in the sample have varying levels of staff size ranging from 2 staffmembersupto127.Thetypeofstaff involvedusually includesmanagerialandadministrativestaff.Toa lesserextent theyalsohaveresearchstaffother staffinvolvedonthecontentlevelwithexecutingEQAprocedures.Whenaskedaboutsatisfaction with the qualifications and sufficiency of the staff members, theagenciesseemtobesatisfiedwithbothbutnoneoftheagenciesstronglyagreedwiththeitems,whichsuggestsfurtherneedinhumanresourcecapacitybuilding.However,interestinglyenough,theagenciesdidnotdiscusstheneedforarobusttrainingplanfor itsQAAstaffmembers, toprepare leadersthatwouldactuallydrivetheQAsystemenhancement.Mostofthemjusthavethestaffaboutrighttoconduct the procedures; fewer of them had research staff (26%) to enable abetterunderstandingofthesystemneedsthusleadingitsenhancement.As for thequalificationsof the staffmembers, someof themhad requirementsfor at least having a BA degree, others set the level to MA degree. Yet otheragencies required MA degree and PhD for some of the positions. One of theagenciesadditionallyrequiredalicense,forGeneralAdministrationAct,whichisastateexamforobtainingalicenseforfunctioninginthefield.Workexperienceinthefieldwasalsosetasoneoftherequirementsinsomecases.Regardingthetopicofhumanresourcecapacity,welookedattheaccessoftheagencies toapoolof competentexpert to conductexternal evaluations. 77.3%responded they had access to such a pool for both national and internationalpeersand27.3%hadsuchapoolonlyfornationalpeers.


18

With regards to the financial capacity, the predominant sources are fromgovernmental allocations (60.8%), evaluation services (60.8%) and grants anddonations(30.4%).VeryfewagenciesstatedhavingsuchincomesourcesasHEIsubscription(1case)andmembershipfees(1case)(seeChart4),mostofthemareconcernedwithunderfunding.

Chart4:EQAAfundingmechanismsThe open‐ended questions revealed that the agencies are in need of financialresources to allocate towards system‐wide research and identification of gapsanddevelopmentaltrends.Not surprisingly, the development and enhancement of EQAAs heavily dependontheavailabilityoffinancialresources,theabsenceofwhichhasapotentialtounderminetheircredibilityleadingtoseriousdamagestothesystem.

5. ExternalEvaluationProceduresWhenwelookedattheexternalqualityassurancemechanismsusedbydifferentsystems, accreditation at institutional and programme levels seems to bedominant. In the open‐ended questions there was a clear expression ofoverabundanceandoverlapofthetoolsinsomeofthesystems.

IR Institutionalrecognition PE ProgrammeevaluationCIT Confermentofinstitutionaltitle PA ProgrammeAccreditation

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00%

Government

Membershipfee

Evaluationservices

Grantsanddonations

HEIsubscriptionfee

Other

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

SAR

Deskreviewandsite‐visit

Panelreport

Follow‐up


19

IA InstitutionalAccreditation C_LLL CertificationofLifeLongLearningI_ISO_C InstitutionalISOcertification TNE_R RecognitionofTransnationalEducationA_FQE Assessment of foreign qualifications

equivalence

Chart5:Qualityassuranceproceduresandmajorphases

Theexternal evaluation isdoneona cyclical basis ranging from5‐7years andpredominantlyevolvesaroundthefollowingmajorsteps:preparationoftheself‐assessmentreport,desk‐review,site‐visit,productionoftheexpertpanelreportand follow‐up on the recommendations. As the chart demonstrates there aredifferent approaches to application of themajor steps to different procedures.Predominantly,forinstitutionalaccreditation(IA)andprogrammeaccreditation(PA)allthemajorstepsareappliedinmostofthecases.However,withregardsto the other tools such as institutional recognition (IR), conferment ofinstitutional title (CIT), institutional ISO certification (I_ISO_C), programmeevaluation(PE),certificationoflifelonglearningmodules(C_LLL),recognitionoftransnational provisions (TNE_R), and assessment of foreign qualificationsequivalence(A‐FQE)thesemajorstepsareappliedatvaryinglevels.Asforthesite‐visit,itismandatoryin100%ofthecasesandthedurationofthesite‐visitisatvaryinglevelsrangingfrom1‐2daysin39%ofthecasesand3‐5daysin56.5%ofthecases,andmorethan5daysintherestofthecases.Generally EQAAs use external evaluation teams for their major evaluations(86.4%)and in13.6%cases the team isnot invited.With regards to thepanelthatconductsexternalevaluation,72.7%stated it is conductedby theexternalteamandEQAArepresentative,4.5%stateditisconductedonlybytheEQAAand27.3%stateditwasconductedonlybytheexternalevaluators.ThebodiesresponsibleforappointingtheexternalevaluationpanelsareEQAAs(85%), the Accreditation Commissions (20%) and to a lesser extent it is alsodonebytheGovernment/Ministry(10%).ThecompositionofthepanelbringsinavarietyofstakeholdersatavaryinglevelasdemonstratedinChart6.


20

Chart6:Compositionofthepanel

Predominantlythestakeholdersinvolvedarestudents(70%ofthecases), localand international experts 90% and 80% respectively as well as professionalassociations (65%). Some of the panels also include employers (50%),professional practitioners (30%), HE administration (50%) and EQAA staffmembers(30%).TheEQAAstaffmembersmainlyactasprocedurecoordinators.As for the distribution of the responsibilities throughout the procedure, theEQAA procedure coordinators are mostly involved in all the steps supportingHEIandprogrammeintheirconductoftheself‐evaluation,guidingthepanelinthedesk‐review,planning thesite‐visit.Toa lesserextent theytakepart inthesite‐visitandreportproduction.

Chart7:RoleofEQAAsandexpertsintheproceduresAsthechartshows,thereisatendencyonthepartofEQAAcoordinatorstobeactive during the preparatory phase while during the site‐visit and reportproduction the external evaluators take a lead. During the desk‐review EQAAstaffhasactiveparticipationin63.64%cases.Asfortheroleoftheexperts,theyareinmostcasesdealingwithdesk‐reviewin72.73% of the cases, site‐visit in 100% of the cases and report production in95.45%of thecases.Toa lesserextent theyalsoparticipate in theplanningofthesite‐visit(59%).Thenextpoint refers to theactivities conductedby thepanelsduring the site‐visit.

0.00%20.00%40.00%60.00%80.00%100.00%

Functionnotperformed

Expertpanel

EQAA


21

Chart8:Activitiesconductedduringthesite‐visit

In practically all cases, such activities as interviews with faculty and staff(100%), reviewofdocumentaryevidence(100%), tourof thecampus facilities(95.5%),meetingwith theHEImanagement topresent thepreliminary results(95.5%),andinterviewwithstudents(90.9%)werestatedtobepartofthesite‐visit.Toalesserextentinterviewwithalumni(63.6%),observationofteachingpractices(63.6%),interviewswithexternalstakeholders(54.5%)andemployers(54.5%)werementioned. It is interestingtonote,thattherearestillcasesthatapplystudentassessmenttests(22.7%),whichwereprevalentduringthepost‐Soviet (pre‐Bologna) HE education system formation, when the functions ofaccreditationstillwereunder the jurisdictionof theMinistryofEducation.Thenature of accreditation in pre‐Bologna system evolved around mainly anoutcome‐based approachwith emphasis on quantitative data and the externalevaluatorsmainlylookedatthestudentexaminationresultsintwoconsecutiveyears to take decision on accreditation. In this case the HEI was awardedaccreditation status if about 65% of students managed to pass the minimumthreshold.Thus,afterthesitevisitoralreadyduringittheexternalevaluatorsdraftareporton the major findings and outline commendations and recommendations forfurther improvement. Once the draft report is ready the external evaluationteamsusuallyconsultwiththeHEIsunderevaluation(90.5%),students(9.5%),andprofessionalbodiesandindustry(4.8%each)priortoitsfinalization.The finaldecision takingon theprocedurerests, inmostof thecases,with theaccreditation commission/assembly/council – the decision taking bodies thatareactuallyactingasalsogoverningbodiesoftheagenciesexceptforsomecaseswhere there is a separate Accreditation Commission responsible for decisiontakingotherthanitsBoardofTrustees.Toalesserextent13.6%theexpertpaneltakes the decision. Considering most of the decision‐taking bodies havegovernment and ministry representatives in it and in some cases the latterpresideover thisbodyex‐officio, independencyof theproceduresanddecisiontakingisoneofthemajorissuesyettobesolved.Thereportsarepublishedinfull in59.1%cases,partly in18.2%casesandnotpublishedatallin22.7%cases.


22

Last,butnotleast,animportantstepintheprocedureisthefollow‐uporganizedby the agency. Primarily, the follow up evolves around follow‐up site‐visit(63.6%), request for a remedial plan (77.3%), and request for a status report(63.6%). To a lesser extent there is an evaluation exercise conducted anew(31.8%).An open‐ended question also looked at the challenges that the agencies arefacing while conducting their external evaluation procedures. The challengesthatwereamongthemostfrequentonesareasfollows:

‐ lack of capacity on the part of the HEIs to produce an analytical self‐evaluation report. It is mainly descriptive in nature providing littlebackgroundforunderstandingthecapacityoftheHEI/programme;

‐ thenewlydevelopingcultureofapeer‐reviewintheareastillencounterslack of qualified local experts to come upwith valuable insights on theHEI/programmefunctioningand,therefore,recommendationsleadingtoenhancement;

‐ follow up procedures are not yet fully in place and are not clearlydescribed;

‐ thelanguageofthereportsproducedbytheexternalevaluatorsisstillachallenge,sinceitmakesunclearwhattherecommendationsarebasedonandwhattheyentail;

‐ under funding of those agencies prevents the latter to research therespectiveenvironments,findoutthegapsanddevelopmentaltrendsandensureproperstrategicdevelopmentofthesystem;italsopreventsfromactive participation at international levels since to be included in anetwork and participate in its activities costs money. Even more, if anexternal recognition against ESG or INQAAHE standards is to beundertaken,thentheagenciesfindthemselvesdeprivedoftheluxury;

‐ compliance with ESGs, becoming ENQA member and thus beingregistered in EQAR is yet another challenge the agencies have to fightwith. Inmostof the cases thegovernmentalarrangementsdonotallowfor this sort of compliance because of the controlling nature of thegovernments;

‐ havingstrongcandidatesasaninternationalpeerisachallengeforsomeof the agencies, since it is difficult to find those that fitted to theprocedureandcontexts;

‐ in some systems, multiplicity of various review procedures leads tosuperficialityofapproachesanddiminishedtrustinQA;

‐ inthesystemswhereQAhasbeenfordecades,thereisanurgentneedtorevisetheapproaches–criteriaandprocedures–tomeettheneeds;

‐ in somesystems, frequentchangesat thepolicy‐making level (legalandregulatory frameworks) make the mechanisms applied inconsistentlyappliedthroughoutthecases;


23

‐ lackofNQFwas alsomentionedas oneof themosturgent issues tobesolved.

‐ Assessmentofstudentlearningoutcomes‐ Involvementofstudentsinevaluationprocedures‐ Mergingofinstitutionalandprogrammeaccreditation‐ Introduction of a comprehensive system for feedback on the agency’s

activities–IQAoftheagencies;‐ The need to revise the programmes to bring in a learning‐outcome

approach.

Thus,asseenfromtheconcernsraisedbydifferentsystems,primarilycapacitybuilding of external reviewers and EQAA staff in conducting procedures,interpretingstandards,evaluatingtheHEIsandprogrammesandcomingupwithrelevantandenhancement‐drivenrecommendationsseemstobecrucialat thisstage.Anothermajorconcernrelatestobuildingonthecapacityofthesystemtoadopt a learning‐outcome approach to programme delivery and understandbetterthemethodsof learningoutcomeassessment.Last,butnotleastconcernrelates to the follow‐up. As the respondents stated, there is a need to betterunderstandrespectivemethodologiesandtheirapplicationinpracticetoensuresystemenhancement.

6. QualityAssessmentCriteria:DevelopmentandImprovementAll the countries inour sample in the three respective regions referred to twomajor sets of criteria applied in the external evaluation – national qualitystandards(91.3%)andESGs(82.6%).In8.7%ofcasesISOwasalsoappliedandin26.1%ofcasestheagenciesappliedeithercriteriadevelopedbytheagenciesthemselvesorsubjectspecificcriteriasuchasWFMEstandards.47.6%oftheagenciesapplyfieldspecificcriteriawhenconductingprogrammeaccreditation or evaluation and 52.4% apply generic criteria for the sameproceduresattheprogrammelevel.Whendevelopingandrevisingtheaccreditationcriteriaalltheagenciesstateditwas done with broad involvement of such stakeholders as students (86.4%),governments (68.2%), tertiary education institutions (90.9%), professionalassociations(63.6%)andrepresentativesofthelabormarket(77.3%).Insomecasesadditionally internationaland localexpertswere involved in theprocess.Whenaskedaboutthemeansofinvolvementtheagenciespredominantlystatedorganization of workshops and conferences. Some agencies in addition statedadministrationofsurveysandroundtablediscussions.


24

Chart9:Involvementofstakeholdersintherevisionofprocedures

Thebodiesthattakedecisionsonthecriteriaareeitherthegovernments/ministries(43.47%),accreditationcouncils(43.47%)ortheagenciesthemselves(13%).An in‐depth analysis of the newly emerging systems has revealed that in theprocess of developing criteria in most of the cases the EQAAs draw oninternational best practice. In some cases those practices are adopted withconsiderationofthelocalneeds.However,inothercasesthesestandardsdonotfind the necessary refinement tomeet the needs of the system, thus, result indamagingratherthanimprovingit.Another major concern is related to the interpretation of standards and thecapacitytoworkwiththosestandards,whichareactuallyattheheartofsuccessofanysystem.

7. AccountabilityMechanismsWealsolookedattheaccountabilitymechanismsappliedbyvariousagencies.Itisgoodtonote,thatalmostallagencieshaveconflictofinterestpolicy, internalqualityassuranceofagenciesandappealssystemasmajoraccountabilitytoolsinplace.Whilemostagenciesarealsoreviewedbynationalauditingbodies,fewerstate compliancewithENQA standards, EQAR and INQAAHEGGP.As Chart 10showssomeagenciesareeitherintheplanningorimplementationphasegettingreadytoberecognizedagainstENQAESGsandINQAAHEGGP.Only7agencies(30.43%) have actually undergone international review against ESGs and thetermofvalidity fortheENQAmembership isalreadyexpiredforoneagencyin2012.ISOisappliedonlyinonecaseandthereisnotangibletrendonthepartofotherstooptforthataccountabilitymechanism.


25

(IQA)internalqualityassuranceofEQAA(CoI)conflictofinterests(NR)nationalreviewbyauditingbodies(GGP)INQAAHEGuidelinesofGoodPractice

(EQAR) EuropeanQualityAssuranceRegister(ENQA) European Network of QualityAssurance(ISO) International Organization forStandardizationAppealssystem

Chart10:Accountabilitymechanismsinpractice

Only in two cases an additional accountability mechanism in the form of aNational Register of QA Agencies, which lists the agencies (local andinternational) that meet the criteria set by the given governments, wasregistered.Thus, the role of agency’s internal quality assurance mechanisms cannot beoveremphasized.ToensureeffectiveandefficientfunctioningtheEQAAsshouldpayparticularattentiontotheestablishmentandoperationalizationof internalqualityassurancesystemsattheagency level.Asforthe internationalvisibilityand credibility the EQAAs undergoing an international recognition procedureagainstcommonlyacceptedstandardsshouldbecomepriorityontheagendaoftheagenciesandrespectivegovernments.

8. StakeholderInvolvementStakeholder involvement is one of the prerequisites for establishment of aqualityculture,sinceitmakestheQAsystemlegitimate,raisesownershipofandcommitmentofthestakeholders,thusleadingtocontinuousenhancementoftheHEsysteminthecountry.The agencies in the sample are involved in the quality assurance activities atvarious levels and to varying extent. Involvement of such stakeholders asstudents, governments, employers, HEIs is expressed through participation inthedecision‐making(91.30%),developmentofproceduresandcriteria(66.7%)throughconsultation,implementationofprocedures(95.65%),capacitybuilding

0.00%10.00%20.00%30.00%40.00%50.00%60.00%70.00%80.00%90.00%100.00%

Fullyestablished

implementationinprocess

beingplanned

plannedwithoutschedule

notplannednotestablished


26

events, conferences (61.9%)workshops (66.7%). Only in 19.0% cases did theagencies mention not taking measure for involvement. Other measures ofinvolvementevolvearoundorganizationofroundtablediscussionsandsurveys.When asked about further potentials for an increased involvement ofstakeholderstheagenciespredominantlystated:

‐ revision of the existing legal frameworks to encourage such an involvement;overallrevisionofHEstrategytounderpintheroleofstakeholderinvolvementinit;

‐ increaseinfundingallowingformoreinteractionwiththestakeholdersthroughmassmedia,betterPRactivities,in‐depthresearchintothesystem‐wideneeds,deliveryofmoreworkshopsandorganizationofroundtablediscussions;

‐ frequentorganizationofcapacitybuildinganddisseminationevents;‐ making the QAA more proactive in their activities with the HEIs and other

stakeholders;‐ conveying the values and set of incentives of quality assurance to the key

stakeholders;‐ includingthestakeholdersinthedecision‐making.

9. CapacityBuildingAs for thecapacitybuilding it takesplaceat theagencystaff level (81.8%), forlocal and international evaluators (85%), IQA staff of HEIs. For the IQA of theHEIs31.8%oftheagenciesrespondedtheyorganizeditseveraltimesperyear,36.4%diditonceperyear,27.3%conductedtrainingsratherseldom,and9.1%didnotconductatall.Themostfrequenttopicsfortrainingtheexternalevaluatorsincluded

‐ approachestoevaluation;‐ legislationandrecentdevelopments;‐ interpretationofcriteriaincludingtheESGrevision;‐ qualityassurance:internalandexternal;‐ approachestoevaluationbasedonsubjectspecificcriteria;‐ goodandbadpracticestoexternalevaluation;‐ internalregulationsoftheagency.

AsfortheHEIsthetopicsevolvedaround‐ introductionofchangestolegislation;‐ introductiontoQAregulationsandprocedures;‐ internalqualityassurance:approachesandmechanisms‐ approachestoself‐assessment‐ externalqualityassurance.


27

10. NetworkingToensureanagency is in linewiththedevelopmentsat the international levelmostly the agencies in our sample tend to be involved in one ormore qualityassurance networks. Almost all the agencies have special staff memberresponsiblefortheEQAA’sactivitieswithinthosenetworks(81%).Mostly, the agencies state the benefits of involving in the regional andinternationalnetworksas thoseofenrichingknowledgeonQAbysharingbestpractices (95.5%), and increased legitimacy and international recognition(86.4%).Asotherbenefitsenhancementof thesystemaswellaspotentials forenrichingtheirpoolsofinternationalexpertswerementioned(9%).As for the nature of participation in those networks the agencies reflected onsuchmodesasparticipationinannualmeetings,participationindevelopmentalactivities, contribution to the newsletter, organization of workshops,involvementintheboardsofthenetworksaswellasintheworkinggroups.

Chart11:NatureofinvolvementininternationalnetworksAsseenfromtheChart,theagenciesmostlyparticipateintheannualeventsandtosomeextenttheycontributetothenewslettersandregulatoryandproceduraldevelopments. Considering such networks tend to have representation of theregionscoveredontheGoverningBoardssuchkindofinvolvementalsoseemstoactivate and encourage respective countries through instilling the sense ofresponsibilityandownership.


28

V. CONCLUSIONSCompared to other higher education systems,which enjoy a history of severaldecades (UK and Ireland) or even centuries (USA) in a systemic approach toexternal and internal quality assurance in higher education, the countries inEasternEurope,Caucasus,CentralAsiaandMongoliahavebeenendeavouringtoestablisharobustsystemofexternalandinternalqualityassuranceonlyforthelast twodecades. The countries in the target regions started to establishEQAsystemswithvaryinggovernancemodelsandextentofsuccessasearlyasatthebeginningofthe1990s.AfterjoiningtheBolognaprocessstartingfrom1999,amajor revision to EQA has been observed, underpinning such concepts asindependence,qualityenhancementand,consequently,establishmentofqualityculture.Currently,thecountriesinthesampleareinthestageofenhancingtheadopted approaches to achieve better integration at the national andinternationallevels.INQAAHE has conducted a scoping study on the state of affairs of the threeregions inNovember 2013‐March 2014. Themajor findings have revealed thefollowing developmental trends in the countries of Eastern Europe, Caucasus,CentralAsiaandMongolia.First, the initiative of establishing EQAs in the target regions came from thegovernments as opposed to thepractices in suchdecentralized systems as theUSA and New Zealand, where the initiators were the HEIs themselves. Whilecommendableonthepartofthegovernmentsandministries’inthesetupoftheEQAs at the initial stage, later on in the development process the extensiveinterference from the governments and ministries seems to impede HEIdevelopment leading to infringement of HEIs autonomy, decreasedaccountabilityand thereforequalityofprovisions. In someof thecountries theEQA is still under the jurisdiction of the Ministry of Education and suchimplementation of EQAquestions its compliancewith internationally acceptedgoodpracticesleadingtothesystemenhancement.Moreover,ontheissuesofgovernanceaswellasindependenceoftheEQAAs,theagencies are still influenced by the participation of ministry and governmentrepresentatives, who, in some of the cases, preside over the decision‐makingbodiesonanex‐officiostatus.Ontheotherhand,someofthegoverningbodiesinclude international experts, which bring in diversity and impartiality indecision‐taking tosomeextent.Thus, independenceofEQAAs’decision‐makingshould beunderpinned to ensure impartiality of decisions and increased trustandcredibilityoftheexternalqualityassurance.Second,NationalQualificationsFrameworksarealreadyestablished insomeofthe countries and are in the making in others. Some systems have alreadylaunched tools measuring and assessing alignment of study/academicprogrammeswiththeNQFs.However,theneedtodevelopandimplementtools


29

ensuring alignment of the study/academic programmes with NQFs as well ascheckingthatalignment,hasnotbeeninitiatedinmostofthecountriesyet.SuchtoolsarenecessarytoattestthevalidityandrelevanceoftheNQFs/SQFsaswellasthestudy/academicprogrammes.Third,capacitybuildingof theEQAAstaffandtheexternalevaluatorsseemstobe essential for organization of credible procedures and production of reportsthat would benefit the systems with their recommendations. Still seriouschangesaretobetakentoclosethequalityassuranceloopbyfurtherimprovingthesystemoffollow‐upontherecommendations,thus leadingtoimprovementandenhancement.Apro‐active involvementof theEQAAstaffandtheexternalevaluators in the activities of the international and regional networks mightserveasoneofthebestwaysofcapacitybuilding.Fourth,financialcapacityoftheEQAAsinthetargetcountriesisatriskbecauseof the underfunding from the government and lack of alternative revenuesources. Some systems manage to generate incomes from such alternativesources as grants and donations, and still others build on this componentthroughsettingmembershipfees.Thus,themanagementoftheagenciesneedsafurther capacity building to enable revision of approaches to funding andgenerationofalternativeincomesotherthangovernmentalallocationsandfeesfromtheevaluationprocedures.Thelatter,accordingtotherespondents,isnotenoughforundertakingresearchanddevelopmentalactivitiesbytheEQAAs.Fifth, involvement of stakeholders seems to be one of the significant featuresacrossthesystems.Theexternalreviewers’panelsincludethekeystakeholdersin most of the cases. In some cases, additionally, international peers are alsorepresented and the procedure is conducted in two languages for increasedrecognition of the procedures at the international level. However, suchmajorimpedimentsasrestrictions in the legal framework, lackof funding, scarcityofcapacitybuildingeventsanddisseminationactivitiesandtoalesserextentlackofinclusioninthedecision‐makingprocesseshavebeenidentified.Last, but not least robust accountability mechanisms for EQAAs are to beestablishedinmostofthesystemsand inthosethatarealreadyestablisheditsfurther improvement is crucial for effectiveness and efficiency of the EQAA’soperations. Such exercise as external recognition of the agency againstinternationallyacceptedstandards(ESGs,GGP,ChibaPrinciples)shouldbetakenseriouslyandregularlyiftheagenciesaretoberecognizedforcredibilityoftheiroperationsatnationalandinternationallevels.Consequently,althoughtherehavebeenseriouschangesinthehighereducationsystems allowing establishment and implementation of independent externalquality assurance systems, further investments are needed to lead to theimprovement,enhancementandincreasedcredibilityinthesystems.


30

VI. APPENDICES

AppendixA:ListofAcronymsAPQN Asia‐PacificQualityNetwork

CEENQA CentralandEasternEuropeanQualityAssurance

CHEA‐CIQG CouncilofHigherEducationAccreditation–CHEAInternational

QualityGroup

EAQAN EurasianQualityAssuranceNetwork

ECA EuropeanConsortiumofAccreditation

ENQA EuropeanNetworkofQualityAssurance

EQA ExternalQualityAssurance

EQAA ExternalQualityAssuranceAgencies

EQANIE EuropeanQualityAssuranceNetworkforInformaticsEducation

EQAR EuropeanQualityAssuranceRegister

ESG StandardsandGuidelinesforQualityAssuranceintheEuropean

HigherEducationArea

GGP GuidelinesofGoodPractice(INQAAHE)

GIQAC GlobalInitiativeonQualityAssuranceCapacity

HEI HigherEducationInstitution

INQAAHE InternationalNetworkofQualityAssuranceAgenciesinHigher

Education

IQA InternalQualityAssurance

ISO InternationalOrganizationofStandardization

NQF NationalQualificationsFramework

QA QualityAssurance

SQF SectorialQualificationsFramework

UNESCO UnitedNationsEducational,ScientificandCulturalOrganization


31

AppendixB:ListofParticipantCountriesandAgencies

Country Sentquestionnairesto Externalrecognitionstatus

Responses

1 Albania PAAHE Yes2 Bosnia and

HerzegovinaHEA YesHEAARS Yes

3 Belarus NoQA agency, the functionisdonebytheMinistry

4 Bulgaria NEAA ENQArecognized

Yes

5 Croatia ASHE ENQArecognized

Yes

6 CzechRepublic

ACCR ENQArecognized

No

7 Estonia EKKA ENQArecognized

Yes

8 Hungary HAC ENQArecognized

Yes

9 Kosovo KAA Yes10 Lithuania SKVC ENQA

recognizedYes

11 Latvia AIKNC Yes12 Montenegro Centre for Studies and

QualityAssurance

No

13 Poland UAC YesPKA ENQA

recognizedNo

14 Romania ARACIS ENQArecognized

Yes

15 Russia NCPA Intheprocess YesNAA ENQA

recognized–membership

expiredin2012

Yes

AKKORK Yes16 Slovakia AC Yes17 Slovenia SQAA Yes18 Ukraine NoQA agency, the function

isdonebytheMinistry

19 Armenia ANQA Proof‐evaluatedbyEUexperts,plannedfor

2015

Yes

20 Turkey TEPDAD Yes


32

21 Moldova NoQA agency, the functionisdonebytheMinistry

No

22 Georgia NCEQE Yes23 Azerbaijan NoQA agency, the function

isdonebytheMinistryNo

24 Mongolia NCEA Yes

25 Kyrgyzstan EDNET,theinitiativeforQAinHEwascontacted

No

26 Kazakhstan IAAR YesIQAA Yes

27 Tajikistan NoQA agency, the functionisdonebytheMinistry

No

28 Uzbekistan NoQA agency, the functionisdonebytheMinistry

No

29 Turkmenistan

NoQA agency, the functionisdonebytheMinistry

No


33

AppendixC:ContactdetailsofParticipatingQAAgencies1 Country Albania

NameofAgency: PublicAccreditationAgencyforHigherEducation

Acronym: PAAHEPhone: +35542257954E‐mail: [email protected]‐site: www.aaal.edu.al/en

Nameofthepersonthatfilledinthequestionnaire:

DhurataBozo

Position: Director2 Country Armenia

NameofAgency: NationalCenterforProfessionalEducationQualityAssurance,Foundation

Acronym: ANQAPhone: +37410229145E‐mail: [email protected]‐site: www.anqa.amNameofthepersonthatfilledinthequestionnaire:

RubenTopchyan

Position: Director3 Country BosniaandHerzegovina

NameofAgency: HigherEducationAccreditationAgencyofRepublicSrpska

Acronym: HEAARSPhone: +38751922081E‐mail: [email protected]‐site: http://www.heaars.com/en/indexen.htmlNameofthepersonthatfilledinthequestionnaire:

MiroslavBobrek

Position: Director4 Country BosniaandHerzegovina

NameofAgency: WUSAustriaforHEAAcronym: HEAPhone: +4369914340404E‐mail: Adi.kovacevic@wus‐austria.orgWeb‐site: http://www.wus‐austria.org/Nameofthepersonthatfilledinthequestionnaire:

AdiKovacevic

Position: Director

mailto:[email protected]

http://www.aaal.edu.al/en


http://www.anqa.am


http://www.heaars.com/en/indexen.html

mailto:Adi.kovacevic@wus%E2%80%90austria.org

http://www.wus%E2%80%90austria.org/


34

5 Country Bulgaria

NameofAgency: NationalEvaluationandAccreditationAgency

Acronym: NEAAPhone: +35928077812E‐mail: [email protected]‐site: http://www.neaa.government.bg/enNameofthepersonthatfilledinthequestionnaire:

MilaPenelova

Position: SecretaryGeneral6 Country Croatia

NameofAgency: AgencyforScienceandHigherEducationAcronym: ASHEPhone: +38516274851E‐mail: [email protected]‐site: https://www.azvo.hr/index.php/en/Nameofthepersonthatfilledinthequestionnaire:

DurdicaDragojevic

Position: ExpertAdvisorforInternationalCooperation7 Country Estonia

NameofAgency: EstonianHigherEducationQualityAssuranceAgency

Acronym: EKKAPhone: +3726400454E‐mail: [email protected]‐site: http://ekka.archimedes.ee/Nameofthepersonthatfilledinthequestionnaire:

MaikiUdam

Position: DevelopmentManager8 Country Georgia

NameofAgency: NationalCenterforEducationQualityEnhancement

Acronym: NCEQEPhone: +995577153301E‐mail: [email protected]‐site: http://eqe.ge/engNameofthepersonthatfilledinthequestionnaire:

NinoKopaleishvili

Position: HeadofInternationalRelationsDivision


http://www.neaa.government.bg/en


https://www.azvo.hr/index.php/en/


http://ekka.archimedes.ee/


http://eqe.ge/eng


35

9 Country HungaryNameofAgency: HungarianAccreditationCommitteeAcronym: HACPhone: +3613440314/121E‐mail: [email protected]‐site: www.mab.huNameofthepersonthatfilledinthequestionnaire:

ChristinaRozsnyai

Position: ProgrammeOfficer,ForeignAffairs10 Country Kazakhstan

NameofAgency: IndependentKazakhstanQualityAssuranceAgencyforEducation

Acronym: IQAAPhone: +77171273822E‐mail: [email protected]‐site: http://www.iqaa.kz/Nameofthepersonthatfilledinthequestionnaire:

SholpanKalanova

Position: President11 Country Kazakhstan

NameofAgency: IndependentAgencyforAccreditationandRating

Acronym: IAARPhone: +77172452402E‐mail: [email protected]‐site: http://www.iaar.kz/Nameofthepersonthatfilledinthequestionnaire:

TimurKanapyanov

Position: ManagerofInternationalProjects12 Country Kosovo

NameofAgency: KosovoAccreditationAgencyAcronym: KAAPhone: +38138213087E‐mail: [email protected]‐site: http://www.akreditimi‐

ks.org/new/index.php/en/Nameofthepersonthatfilledinthequestionnaire:

FerdijeZhushiEtemi

Position: PresidentoftheAccreditationBoard(NationalQualityCouncil)


http://www.mab.hu


http://www.iqaa.kz/


http://www.iaar.kz/


http://www.akreditimi%E2%80%90ks.org/new/index.php/en/




36

13 Country LatviaNameofAgency: Foundation“HigherEducationQuality

EvaluationCenter”Acronym: HEQECPhone: +37129283214E‐mail: [email protected];[email protected]‐site: http://www.aiknc.lv/enNameofthepersonthatfilledinthequestionnaire:

JurisDzelme

Position: ChairmanoftheBoard14 Country Lithuania

NameofAgency: CenterforQualityAssessmentinHEAcronym: SKVCPhone: +37061235339E‐mail: [email protected]‐site: http://www.skvc.lt/enNameofthepersonthatfilledinthequestionnaire:

NoraSkaburskiene

Position: ActingDirector15 Country Mongolia

NameofAgency: MongolianNationalCouncilforEducationAccreditation

Acronym: MNCEAPhone: +97670109391E‐mail: [email protected]‐site: www.accmon.mn/enNameofthepersonthatfilledinthequestionnaire:

JamtsTungalag

Position: SeniorOfficerforInstitutionalAccreditation16 Country Poland

NameofAgency: UniversityAccreditationCommissionAcronym: UACPhone: +48510249543E‐mail: [email protected]‐site: http://www.uka.amu.edu.pl/english.phpNameofthepersonthatfilledinthequestionnaire:

SbigniewPalka

Position: Professor



http://www.aiknc.lv/en


http://www.skvc.lt/en


http://www.accmon.mn/en


http://www.uka.amu.edu.pl/english.php


37

17 Country RomaniaNameofAgency: RomanianAgencyforQualityAssurancein

HigherEducationAcronym: ARACISPhone: +40758075858E‐mail: [email protected]‐site: http://www.aracis.ro/nc/en/aracis/Nameofthepersonthatfilledinthequestionnaire:

MihaiFloroiu

Position: HeadofInternationalRelationsService18 Country RussianFederation

NameofAgency: AgencyforQualityAssuranceinHigherEducationandCareerDevelopment

Acronym: AKKORKPhone: +79037236467E‐mail: [email protected]‐site: http://www.akkork.ru/e/Nameofthepersonthatfilledinthequestionnaire:

ErikaSoboleva

Position: HeadofInternationalRelations19 Country RussianFederation

NameofAgency: NationalAccreditationAgencyAcronym: NAAPhone: +74951346886E‐mail: [email protected]‐site: www.nica.ruNameofthepersonthatfilledinthequestionnaire:

TatianaBarhatova

Position: Director20 Country RussianFederation

NameofAgency: NationalCenterforPublicAccreditationAcronym: NCPAPhone: +78362421640E‐mail: [email protected]‐site: www.ncpa.ruNameofthepersonthatfilledinthequestionnaire:

GalinaMotova

Position: DeputyDirector


http://www.aracis.ro/nc/en/aracis/


http://www.akkork.ru/e/


http://www.nica.ru


http://www.ncpa.ru


38

21 Country SlovakiaNameofAgency: AccreditationCommission,AdvisoryBodyof

theGovernmentoftheSlovakRepublicAcronym: ACPhone: +421259374325E‐mail: [email protected]‐site: http://www.akredkom.sk/en/Nameofthepersonthatfilledinthequestionnaire:

LuborFisera

Position: Chairman22 Country Slovenia

NameofAgency: SlovenianQualityAssuranceAgencyAcronym: NAKVISPhone: +38640636923E‐mail: [email protected]‐site: http://www.nakvis.siNameofthepersonthatfilledinthequestionnaire:

IvanLeban

Position: Director23 Country Turkey

NameofAgency: AssociationforEvaluationandAccreditationofMedicalEducationPrograms

Acronym: TEPDADPhone: +905323632634E‐mail: [email protected]‐site:Nameofthepersonthatfilledinthequestionnaire:

H.IbrahimDurak

Position: SecretaryGeneral


http://www.akredkom.sk/en/


http://www.nakvis.si



39

INQAAHESecretariatCatalanUniversityQualityAssuranceAgency,AQUCatalunya

C.delsVergós,36‐42,08017BarcelonaTel:+34932688950

e‐mail:[email protected]


Quality assurance in higher education in Easter Europe,...

Documents

Transcript of Quality assurance in higher education in Easter Europe,...