SABER – SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS STUDENT ASSESSMENT 13 99368 The Development of the Student Assessment System in the Republic of Armenia: Achievements, Challenges, and Lessons Learned George Bethell and Karine Harutyunyan SABER—SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS STUDENT ASSESSMENT 13ȱ TheȱDevelopmentȱofȱtheȱ StudentȱAssessmentȱSystemȱ inȱtheȱRepublicȱofȱArmenia:ȱ Achievements,ȱChallenges,ȱ andȱLessonsȱLearnedȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ ȱ ȱ ȱ ȱ ȱ ȱ ȱ ȱ ȱ ȱ ©ȱ2015ȱTheȱInternationalȱBankȱforȱReconstructionȱandȱDevelopmentȱ/ȱ ȱ TheȱWorldȱBankȱ 1818ȱHȱStreetȱNWȱ WashingtonȱDCȱ20433ȱ Telephone:ȱ202Ȭ473Ȭ1000ȱ Internet:ȱwww.worldbank.orgȱ ȱ 1ȱ2ȱ3ȱ4ȱ 15ȱ14ȱ13ȱ12ȱ ȱ ȱ ThisȱworkȱisȱaȱproductȱofȱtheȱstaffȱofȱTheȱWorldȱBankȱwithȱexternalȱ contributions.ȱTheȱfindings,ȱinterpretations,ȱandȱconclusionsȱexpressedȱinȱ thisȱworkȱdoȱnotȱnecessarilyȱreflectȱtheȱviewsȱofȱTheȱWorldȱBank,ȱitsȱ BoardȱofȱExecutiveȱDirectors,ȱorȱtheȱgovernmentsȱtheyȱrepresent.ȱ TheȱWorldȱBankȱdoesȱnotȱguaranteeȱtheȱaccuracyȱofȱtheȱdataȱincludedȱ inȱthisȱwork.ȱTheȱboundaries,ȱcolors,ȱdenominations,ȱandȱotherȱ informationȱshownȱonȱanyȱmapȱinȱthisȱworkȱdoȱnotȱimplyȱanyȱjudgmentȱ onȱtheȱpartȱofȱTheȱWorldȱBankȱconcerningȱtheȱlegalȱstatusȱofȱanyȱterritoryȱ orȱtheȱendorsementȱorȱacceptanceȱofȱsuchȱboundaries.ȱ ȱ RightsȱandȱPermissionsȱ Theȱmaterialȱinȱthisȱworkȱisȱsubjectȱtoȱcopyright.ȱBecauseȱTheȱWorldȱBankȱ encouragesȱdisseminationȱofȱitsȱknowledge,ȱthisȱworkȱmayȱbeȱ reproduced,ȱinȱwholeȱorȱinȱpart,ȱforȱnoncommercialȱpurposesȱasȱlongȱasȱ fullȱattributionȱtoȱthisȱworkȱisȱgiven.ȱ Anyȱqueriesȱonȱrightsȱandȱlicenses,ȱincludingȱsubsidiaryȱrights,ȱ shouldȱbeȱaddressedȱtoȱtheȱOfficeȱofȱtheȱPublisher,ȱTheȱWorldȱBank,ȱ1818ȱ HȱStreetȱNW,ȱWashington,ȱDCȱ20433,ȱUSA;ȱfax:ȱ202Ȭ522Ȭ2422;ȱeȬmail:ȱ pubrights@worldbank.org.ȱ ȱ Coverȱdesign:ȱPatriciaȱHord.GraphikȱDesign,ȱAlexandria,ȱVAȱ ȱ ȱ ȱ Contentsȱ AboutȱtheȱSeriesȱ.................................................................................................ȱ v Abstractȱ............................................................................................................ȱvii AboutȱtheȱAuthorsȱ ix ............................................................................................ȱ Acknowledgmentsȱ xi ............................................................................................ȱ AbbreviationsȱandȱAcronymsȱ......................................................................ȱxiii ExecutiveȱSummaryȱ........................................................................................xv Introductionȱ ........................................................................................................ȱ1 OverviewȱofȱtheȱStudentȱAssessmentȱSystemȱ...............................................ȱ5 ClassroomȱAssessmentȱ...................................................................................ȱ10 Examinationsȱ....................................................................................................ȱ15 LargeȬScaleȱAssessmentsȱ................................................................................ȱ26 DiscussionȱandȱLessonsȱLearnedȱ ...................................................................ȱ32 Conclusionȱ........................................................................................................ȱ40 Referencesȱ.........................................................................................................ȱ41 ȱ Boxesȱ Boxȱ1:ȱOverviewȱofȱtheȱGradesȱ4,ȱ9,ȱandȱ12ȱstateȱgraduationȱ examinationsȱ............................................................................................ȱ17 Boxȱ2:ȱOverviewȱofȱtheȱUnifiedȱExaminationȱ..............................................ȱ20 Boxȱ3:ȱOverviewȱofȱtheȱstatus,ȱfunctions,ȱandȱresourcesȱofȱtheȱATCȱ.......ȱ22 ȱ Figuresȱ Figureȱ1:ȱStructureȱofȱtheȱArmenianȱEducationȱSystemȱasȱofȱ2015ȱ.............ȱ4 Figureȱ2:ȱAnnualȱenrolmentȱdataȱforȱtheȱfinalȱyearȱofȱgeneralȱ secondaryȱeducationȱ(G11ȱinȱ2010/2011ȱandȱG12ȱthereafter).ȱ...........ȱ39 ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ iiiȱ ȱ ȱ ȱ ȱ AboutȱtheȱSeriesȱ Buildingȱ strongȱ educationȱsystemsȱ thatȱ promoteȱ learningȱisȱfundamentalȱ toȱ developmentȱ andȱ economicȱ growth.ȱ Overȱ theȱ pastȱ fewȱ years,ȱ asȱ developingȱ countriesȱ haveȱ succeededȱ inȱ buildingȱ moreȱ classrooms,ȱ andȱ gettingȱmillionsȱmoreȱchildrenȱintoȱschool,ȱtheȱeducationȱcommunityȱhasȱ begunȱ toȱ activelyȱ embraceȱ theȱ visionȱ ofȱ measurableȱ learningȱ forȱ allȱ childrenȱ inȱ school.ȱ However,ȱ learningȱ dependsȱ notȱ onlyȱ onȱ resourcesȱ investedȱ inȱ theȱ schoolȱ system,ȱ butȱ alsoȱ onȱ theȱ qualityȱ ofȱ theȱ policiesȱ andȱ institutionsȱ thatȱ enableȱ theirȱ useȱ andȱ onȱ howȱ wellȱ theȱ policiesȱ areȱ implemented.ȱ Inȱ 2011,ȱ theȱ Worldȱ Bankȱ Groupȱ launchedȱ Educationȱ Sectorȱ Strategyȱ 2020:ȱLearningȱforȱAll,ȱwhichȱoutlinesȱanȱagendaȱforȱachievingȱ“Learningȱ forȱ All”ȱ inȱ theȱ developingȱ worldȱ overȱ theȱ nextȱ decade.ȱ Toȱ supportȱ implementationȱofȱtheȱstrategy,ȱtheȱWorldȱBankȱcommencedȱaȱmultiȬyearȱ programȱ toȱ supportȱ countriesȱ inȱ systematicallyȱ examiningȱ andȱ strengtheningȱ theȱ performanceȱ ofȱ theirȱ educationȱ systems.ȱ Thisȱ evidenceȬbasedȱ initiative,ȱ calledȱ SABERȱ (Systemsȱ Approachȱ forȱ Betterȱ Educationȱ Results),ȱ isȱ buildingȱ aȱ toolkitȱ ofȱ diagnosticsȱ forȱ examiningȱ educationȱ systemsȱ andȱ theirȱ componentȱ policyȱ domainsȱ againstȱ globalȱ standards,ȱ bestȱ practices,ȱ andȱ inȱ comparisonȱ withȱ theȱ policiesȱ andȱ practicesȱ ofȱ countriesȱ aroundȱ theȱ world.ȱ Byȱ leveragingȱ thisȱ globalȱ knowledge,ȱ SABERȱ fillsȱ aȱ gapȱ inȱ theȱ availabilityȱ ofȱ dataȱ andȱ evidenceȱ onȱ whatȱ mattersȱ mostȱ toȱ improveȱ theȱ qualityȱ ofȱ educationȱ andȱ achievementȱ ofȱbetterȱresults.ȱ ȱ SABERȬStudentȱ Assessment,ȱ oneȱ ofȱ theȱ systemsȱ examinedȱ withinȱ theȱ SABERȱ program,ȱ hasȱ developedȱ toolsȱ toȱ analyzeȱ andȱ benchmarkȱ studentȱassessmentȱpoliciesȱandȱsystemsȱaroundȱtheȱworld,ȱwithȱtheȱgoalȱ ofȱ promotingȱ strongerȱ assessmentȱ systemsȱ thatȱ contributeȱ toȱ improvedȱ educationȱ qualityȱ andȱ learningȱ forȱ all.ȱ Toȱ helpȱ exploreȱ theȱ stateȱ ofȱ knowledgeȱ inȱ theȱ area,ȱ theȱ SABERȬStudentȱ Assessmentȱ teamȱ invitedȱ leadingȱ academics,ȱ assessmentȱ experts,ȱ andȱ practitionersȱ fromȱ developingȱ andȱ industrializedȱ countriesȱ toȱ comeȱ togetherȱ toȱ discussȱ assessmentȱ issuesȱ relevantȱ forȱ improvingȱ educationȱ qualityȱ andȱ learningȱ outcomes.ȱ Theȱ papersȱ andȱ caseȱ studiesȱ onȱ studentȱ assessmentȱ inȱ thisȱ seriesȱ areȱ theȱ resultȱ ofȱ thoseȱ conversationsȱ andȱ theȱ underlyingȱ research.ȱ Priorȱ toȱ publication,ȱ allȱ ofȱ theȱ papersȱ benefitedȱ fromȱ aȱ rigorousȱ reviewȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ vȱ process,ȱ whichȱ includedȱ commentsȱ fromȱ Worldȱ Bankȱ staff,ȱ academics,ȱ developmentȱpractitioners,ȱandȱcountryȱassessmentȱexperts.ȱ Allȱ SABERȬStudentȱ Assessmentȱ papersȱ inȱ thisȱ seriesȱ wereȱ madeȱ possibleȱ byȱ supportȱ fromȱ theȱ Russiaȱ Educationȱ Aidȱ forȱ Developmentȱ (READ)ȱ Trustȱ Fund.ȱ READȱ Trustȱ Fundȱ isȱ aȱ collaborationȱ betweenȱ theȱ Russianȱ Federationȱ andȱ theȱ Worldȱ Bankȱ thatȱ supportsȱ theȱ improvementȱ ofȱ studentȱ learningȱ outcomesȱ inȱ lowȬincomeȱ countriesȱ throughȱ theȱ developmentȱofȱrobustȱstudentȱassessmentȱsystems.ȱ ȱ Papersȱinȱthisȱseriesȱrepresentȱtheȱindependentȱviewsȱofȱtheȱauthors.ȱ ȱ ȱ ȱ ȱ ȱ viȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ Abstractȱ Inȱ orderȱ toȱ maximizeȱ effectiveness,ȱ everyȱ nationalȱ educationȱ systemȱ needsȱ aȱ comprehensiveȱ andȱ coherentȱ approachȱ toȱ studentȱ assessment.ȱ Theȱ SABERȱ frameworkȱ forȱ studentȱ assessmentȱ identifiesȱ fourȱ keyȱ elementsȱ inȱ thisȱ regard:ȱ classroomȱ assessment,ȱ examinations,ȱ nationalȱ largeȬscaleȱ assessment,ȱ andȱ internationalȱ largeȬscaleȱ assessment.ȱ Armeniaȱ hasȱ undoubtedlyȱ madeȱ significantȱ progressȱ inȱ allȱ ofȱ theseȱ areasȱ sinceȱ gainingȱ independenceȱ fromȱ theȱ Sovietȱ Unionȱ inȱ 1991.ȱ Theȱ purposeȱ ofȱ thisȱ paperȱ isȱ toȱ describeȱ Armeniaȇsȱ experiences,ȱ shareȱ theȱ lessonsȱ learnedȱalongȱtheȱway,ȱandȱpointȱoutȱtheȱchallengesȱthatȱcountriesȱfaceȱinȱ tryingȱtoȱensureȱthatȱtheirȱassessmentȱsystemsȱremainȱfitȱforȱpurposeȱinȱaȱ changingȱenvironment.ȱ ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ viiȱ ȱ ȱ ȱ AboutȱtheȱAuthorsȱ Georgeȱ Bethellȱ isȱ anȱ independentȱ consultantȱ specializingȱ inȱ theȱ policyȱ andȱ practicesȱ ofȱ educationalȱ assessment.ȱ Havingȱ startedȱ hisȱ careerȱ asȱ aȱ physicsȱ teacherȱ inȱ aȱ stateȱ secondaryȱ schoolȱ inȱ theȱ Unitedȱ Kingdom,ȱ heȱ subsequentlyȱservedȱasȱaȱsubjectȱspecialistȱatȱtheȱUniversityȱofȱCambridgeȱ schools’ȱ examinationȱ boardȱ where,ȱ amongstȱ otherȱ things,ȱ heȱ wasȱ responsibleȱ forȱ theȱ preparationȱ andȱ scoringȱ ofȱ scienceȱ examinationȱ papers,ȱ andȱ forȱ standardsȱ setting.ȱ Havingȱ becomeȱ involvedȱ inȱ internationalȱassessmentȱwhilstȱinȱCambridge,ȱGeorgeȱhas,ȱforȱmoreȱthanȱ 30ȱ years,ȱ providedȱ consultancyȱ andȱ trainingȱ servicesȱ toȱ governments,ȱ internationalȱ developmentȱ agencies,ȱ andȱ examiningȱ authoritiesȱ aroundȱ theȱ world.ȱ Hisȱ specialȱ interestsȱ includeȱ highȬstakeȱ examinationsȱ andȱ universityȱ admissionȱ systems,ȱ andȱ sampleȬbasedȱ surveysȱ ofȱ studentȱ achievement.ȱ Sinceȱ 1990,ȱ heȱ hasȱ contributedȱ toȱ theȱ reformȱ ofȱ traditionalȱ assessmentȱ practicesȱ inȱ moreȱ thanȱ aȱ dozenȱ ofȱ theȱ transitionȱ countriesȱ ofȱ Southern,ȱ Central,ȱ andȱ Easternȱ Europe.ȱ Heȱ hasȱ coȬauthoredȱ articlesȱ documentingȱ majorȱ assessmentȱ reformsȱ inȱ Slovenia,ȱ Latvia,ȱ Lithuania,ȱ andȱArmenia.ȱFromȱ2005ȱtoȱ2010,ȱheȱledȱtheȱtechnicalȱteamȱthatȱsupportedȱ theȱ successfulȱ implementationȱ ofȱ aȱ centralizedȱ universityȱ admissionsȱ examinationȱ inȱ Armeniaȱ asȱ wellȱ asȱ theȱ establishmentȱ ofȱ theȱ Assessmentȱ andȱTestingȱCenter,ȱwhichȱnowȱservesȱasȱArmeniaȇsȱexecutiveȱagencyȱforȱ schoolȱexaminationsȱandȱlargeȬscaleȱassessmentsȱofȱstudentȱachievement. ȱ Karineȱ Harutyunyanȱ isȱ aȱ Deputyȱ Ministerȱ inȱ theȱ Armenianȱ Ministryȱ ofȱ Educationȱ andȱ Scienceȱ (MOES).ȱ Sheȱ holdsȱ aȱ PhDȱ inȱ Physicsȱ andȱ Mathematics,ȱandȱaȱMaster’sȱdegreeȱinȱSocialȱSciencesȱfromȱYerevanȱStateȱ Universityȱ where,ȱ inȱ parallelȱ withȱ herȱ responsibilitiesȱ inȱ theȱ MOES,ȱ sheȱ teachesȱ andȱ doesȱ research.ȱ Sheȱ hasȱ publishedȱ inȱ theȱ regionȱ ofȱ 30ȱ articlesȱ andȱ books.ȱ Formerly,ȱ Karineȱ servedȱ asȱ Directorȱ ofȱ theȱ Centerȱ forȱ Educationȱ Projects,ȱ overseeingȱ projectsȱ inȱ theȱ generalȱ andȱ higherȱ educationȱ sectors,ȱ includingȱ thoseȱ supportedȱ byȱ theȱ Worldȱ Bank.ȱ Inȱ thisȱ roleȱ sheȱ wasȱ responsibleȱ forȱ theȱ implementationȱ ofȱ theȱ firstȱ Educationȱ Qualityȱ andȱ Relevanceȱ Project,ȱ which,ȱ asȱ describedȱ inȱ thisȱ caseȱ study,ȱ includedȱcomponentsȱdesignedȱtoȱreformȱclassroomȱassessmentȱpractices,ȱ introduceȱaȱ centralizedȱ examinationȱatȱtheȱ interfaceȱbetweenȱ schoolsȱandȱ institutionsȱ ofȱ higherȱ education,ȱ andȱ establishȱ theȱ nationalȱ ATC.ȱ Sheȱ hasȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ ixȱ alsoȱ servedȱ asȱ anȱ independentȱ consultantȱ for,ȱ amongstȱ others,ȱ theȱ Worldȱ Bank,ȱ theȱ Asianȱ Developmentȱ Bank,ȱ andȱ UNESCO.ȱ Herȱ mainȱ areasȱ ofȱ expertiseȱ areȱ educationȱ policy,ȱ analysisȱ andȱ strategicȱ planning,ȱ theȱ financingȱofȱeducation,ȱandȱprojectȱdesignȱandȱevaluation.ȱ ȱ ȱ ȱ ȱ ȱ ȱ ȱ xȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ Acknowledgmentsȱ Theȱ authorsȱ areȱ gratefulȱ toȱ theȱ READȱ Trustȱ Fundȱ teamȱ forȱ theȱ opportunityȱ toȱ presentȱ Armenia’sȱ experienceȱ inȱ assessingȱ studentȱ achievementȱasȱpartȱofȱtheȱSABERȱWorkingȱPaperȱseries.ȱ ȱ Thanksȱ areȱ alsoȱ extendedȱ toȱ Arsenȱ Baghdasaryan,ȱ Deputyȱ Directorȱ ofȱ theȱ Assessmentȱ andȱ Testingȱ Centerȱ andȱ Nationalȱ Researchȱ CoordinatorȱforȱTIMSSȱinȱArmeniaȱforȱhisȱgenerousȱcollaboration.ȱ Specialȱ thanksȱ areȱ dueȱ toȱ theȱ Worldȱ Bankȱ reviewȱ team,ȱ Margueriteȱ Clarke,ȱChelseaȱCoffin,ȱandȱCristianȱAedoȱforȱtheirȱinsightfulȱsuggestionsȱ onȱimprovingȱtheȱcaseȱstudyȱandȱtheirȱcloseȱattentionȱtoȱdetailȱthroughoutȱ theȱeditorialȱprocess.ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ xiȱ ȱ ȱ ȱ ȱ ȱ AbbreviationsȱandȱAcronymsȱ AMDȱ ArmenianȱDramȱ ATCȱ AssessmentȱandȱTestingȱCenterȱ CFEPȱ CenterȱforȱEducationȱProjectsȱ CPDȱ ContinuousȱProfessionalȱDevelopmentȱ EQRPȱIȱ FirstȱEducationȱQualityȱandȱRelevanceȱProjectȱ GDPȱ GrossȱDomesticȱProductȱ GORAȱ GovernmentȱofȱtheȱRepublicȱofȱArmeniaȱ HEIȱ HigherȱEducationȱInstitutionsȱ ICTȱ InformationȱandȱCommunicationsȱTechnologyȱ IEAȱ InternationalȱAssociationȱforȱtheȱEvaluationȱofȱEducationalȱ Achievementȱ ILSAȱ InternationalȱLargeȬScaleȱAssessmentȱ MOESȱ MinistryȱofȱEducationȱandȱScienceȱ NAEPȱ NationalȱAssessmentȱofȱEducationalȱProgressȱ NCETȱ NationalȱCenterȱforȱEducationalȱTechnologiesȱ NIEȱ NationalȱInstituteȱforȱEducationȱ NLSAȱ NationalȱLargeȬScaleȱAssessmentȱ OECDȱ OrganizationȱforȱEconomicȱCooperationȱandȱDevelopmentȱ PIRLSȱ ProgressȱinȱInternationalȱReadingȱLiteracyȱStudyȱ PISAȱ ProgramȱforȱInternationalȱStudentȱAssessmentȱ READȱ RussiaȱEducationȱAidȱforȱDevelopmentȱ SABERȱ SystemsȱApproachȱforȱBetterȱEducationȱResultsȱ TIMSSȱ TrendsȱinȱInternationalȱMathematicsȱandȱScienceȱStudyȱ UEȱ UnifiedȱExaminationȱ USDȱ UnitedȱStatesȱDollarȱ USSRȱ UnionȱofȱSovietȱSocialistȱRepublicsȱ VETȱ VocationalȱEducationȱandȱTrainingȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ xiiiȱ ȱ ȱ ȱ ȱ ȱ ExecutiveȱSummaryȱ Inȱ 1991,ȱ Armeniaȱ emergedȱ fromȱ theȱ breakupȱ ofȱ theȱ Unionȱ ofȱ Sovietȱ Socialistȱ Republicsȱ (USSR)ȱ asȱ aȱ small,ȱ relativelyȱ poor,ȱ independentȱ stateȱ facingȱ significantȱ socioȬeconomicȱ challenges.ȱ Sinceȱ thatȱ time,ȱ itȱ isȱ widelyȱ acknowledgedȱ thatȱ theȱ countryȱ hasȱ successfullyȱ builtȱ aȱ comprehensiveȱ andȱcoherentȱstudentȱassessmentȱsystemȱtoȱcomplementȱwiderȱreformsȱinȱ education.ȱThisȱcaseȱstudyȱdescribesȱhowȱlegacyȱpracticesȱfromȱtheȱSovietȱ periodȱwereȱtransformed.ȱ Progressȱ hasȱ beenȱ madeȱ inȱ theȱ areasȱ ofȱ classroomȱ assessment,ȱ examinations,ȱ andȱ largeȬscaleȱ surveysȱ ofȱ studentȱ achievementȱ atȱ theȱ nationalȱ andȱ internationalȱ levels.ȱ Atȱ theȱ centerȱ ofȱ thisȱ hasȱ beenȱ theȱ establishmentȱ ofȱ aȱ fullyȬfunctioningȱ Assessmentȱ andȱ Testingȱ Centreȱ (ATC)ȱ toȱ serveȱ asȱ theȱ nationalȱ centerȱ ofȱ excellenceȱ forȱ educationalȱ assessment.ȱ Primarilyȱ establishedȱ toȱ developȱ andȱ administerȱ aȱ new,ȱ highȬstakesȱ Unifiedȱ Examinationȱ (UE)ȱ atȱ theȱ school/universityȱ interface,ȱ theȱ ATCȱ hasȱ graduallyȱ assumedȱ responsibilityȱ forȱ otherȱ schoolȱ examinations,ȱ nationalȱ assessments,ȱ andȱ Armenia’sȱ participationȱ inȱ theȱ TrendsȱinȱInternationalȱMathematicsȱandȱScienceȱStudyȱ(TIMSS).ȱ Inȱ theȱ fieldȱ ofȱ classroomȱ assessment,ȱ theȱ leadȱ agenciesȱ areȱ theȱ Ministryȱ ofȱ Educationȱ andȱ Scienceȱ (MOES)ȱandȱ theȱNationalȱ Instituteȱ forȱ Educationȱ(NIE).ȱBuildingȱonȱactivitiesȱtoȱintroduceȱformativeȱassessmentȱ practicesȱ inȱ schoolsȱ carriedȱ outȱ underȱ theȱ Worldȱ BankȬsupportedȱ Firstȱ Educationȱ QualityȱandȱRelevanceȱ Projectȱ (EQRPȱ I),ȱ theȱ MOESȱ hasȱ issuedȱ teacherȬfriendlyȱ regulationsȱ onȱ classroomȱ assessment,ȱ withȱ teachersȱ beingȱ furtherȱsupportedȱ throughȱ methodologicalȱ guidelinesȱ preparedȱ byȱ theȱNIE.ȱ Thisȱ caseȱ studyȱ discussesȱ theȱ driversȱ andȱ mechanismsȱ involvedȱ inȱ pushingȱforwardȱonȱsoȱmanyȱfronts.ȱTheȱkeyȱdriverȱwasȱandȱremainsȱtheȱ Government’sȱ commitmentȱ toȱ improvingȱ educationȱ inȱ orderȱ toȱ strengthenȱ theȱ country’sȱ mostȱ valuableȱ economicȱ resource—itsȱ people.ȱ Thisȱ canȱ beȱ seenȱ inȱ itsȱ willingnessȱ toȱ investȱ heavily,ȱ notȱ justȱ inȱ infrastructure,ȱ butȱ alsoȱ inȱ theȱ ‘softer’ȱ areasȱ ofȱ education,ȱ suchȱ asȱ theȱ developmentȱ ofȱ modernȱ subjectȱ curricula,ȱ learningȱ standards,ȱ andȱ assessmentȱ practices.ȱ Externalȱ supportȱ andȱ adviceȱ fromȱ theȱ Worldȱ Bankȱ addedȱ momentumȱ andȱ ensuredȱ thatȱ noȱ aspectȱ ofȱ assessmentȱ wasȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ xvȱ neglectedȱ inȱ theȱ designȱ andȱ implementationȱ ofȱ projectsȱ facilitatedȱ byȱ creditȱagreements.ȱ Withȱ respectȱ toȱ theȱ developmentȱ ofȱ theȱ UE,ȱ aȱ secondȱ driverȱ commonlyȱ foundȱ inȱ formerȬUSSRȱ countriesȱ addedȱ momentum—theȱ desireȱ toȱ fightȱ corruptionȱ inȱ theȱ universityȱ admissionsȱ system.ȱ Inȱ short,ȱ theȱintroductionȱofȱtheȱUEȱwasȱseenȱasȱanȱopportunityȱtoȱincreaseȱfairnessȱ andȱtransparencyȱandȱtoȱreduceȱmalpractice.ȱ ȱ Finally,ȱ theȱ caseȱ studyȱ highlightsȱ threeȱ keyȱ lessonsȱ learnedȱ whilstȱ modernizingȱ assessmentȱ practicesȱ inȱ Armenia,ȱ whichȱ areȱ likelyȱ toȱ beȱ ofȱ valueȱtoȱthoseȱstillȱonȱtheȱroadȱtoȱreform.ȱTheseȱare:ȱ x Theȱ technicalȱ elementsȱ ofȱ newȱ assessmentsȱ areȱ relativelyȱ easyȱ toȱ introduce;ȱ itȱ isȱ farȱ moreȱ difficultȱ toȱ ensureȱ thatȱ theȱ informationȱ theyȱ generateȱ isȱ usedȱ effectively.ȱ Assessmentȱ specialistsȱ mustȱ takeȱ theȱ leadȱ inȱ ensuringȱ thatȱ policyȱ makersȱ understandȱ theȱ potentialȱ powerȱ ofȱ assessmentȬrelatedȱ data.ȱ Theyȱ alsoȱ haveȱ anȱ importantȱ roleȱ toȱ playȱ inȱ ensuringȱ thatȱ teachersȱ getȱ theȱ informationȱ theyȱ need,ȱ inȱ formatsȱ theyȱ understand,ȱ inȱ orderȱ toȱ improveȱteachingȱandȱlearning.ȱ x Inȱ theȱ caseȱ ofȱ examinations,ȱ whereȱ theȱ stakesȱ areȱ highȱ andȱ theȱ publicȱ perceptionȱ ofȱ corruptionȱ isȱ great,ȱ thereȱ isȱ inevitablyȱ aȱ tensionȱ betweenȱ theȱ needȱ toȱ maintainȱ examinationȱ securityȱ andȱ theȱ desireȱ toȱ promoteȱ importantȱ educationalȱ objectivesȱ throughȱ moreȱinnovativeȱandȱeffectiveȱapproachesȱtoȱexaminationȱdesign.ȱ Achievingȱ anȱ acceptableȱ andȱ effectiveȱ compromiseȱ isȱ aȱ constantȱ challenge.ȱ x Assessmentsȱ areȱ notȱ isolatedȱ fromȱ otherȱ aspectsȱ ofȱ society.ȱ Asȱ policies,ȱpriorities,ȱandȱotherȱcontextualȱfactorsȱchange,ȱthereȱwillȱ beȱ aȱ continualȱ needȱ toȱ reviewȱ theȱ ‘fitnessȱ forȱ purpose’ȱ ofȱ assessmentȱpracticesȱand,ȱwhereȱnecessary,ȱadaptȱthem.ȱInȱshort,ȱ inȱ studentȱ assessment,ȱ asȱ elsewhere,ȱ theȱ reformȱ processȱ isȱ continuousȱandȱwithoutȱend.ȱ ȱ ȱ ȱ xviȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ TheȱDevelopmentȱofȱtheȱStudentȱ AssessmentȱSystemȱinȱtheȱ RepublicȱofȱArmenia:ȱ Achievements,ȱChallenges,ȱandȱ LessonsȱLearnedȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ Introductionȱ Inȱ orderȱ toȱ beȱ effective,ȱ aȱ nationȇsȱ educationȱ systemȱ mustȱ incorporateȱ technicallyȱ soundȱ studentȱ assessmentȱ practicesȱ servingȱ aȱ wideȱ rangeȱ ofȱ purposes.ȱ Theseȱ purposesȱ includeȱ notȱ onlyȱ theȱ certificationȱ ofȱ studentȱ achievementȱ(i.e.,ȱassessmentȱofȱlearning),ȱbutȱalsoȱtheȱpromotionȱofȱbetterȱ teachingȱ andȱ learningȱ (i.e.,ȱ assessmentȱ forȱ andȱ asȱ learning)ȱ andȱ theȱ provisionȱ ofȱ objectivelyȬverifiableȱ evidenceȱ toȱ informȱ policyȱ makingȱ andȱ strategicȱ planningȱ (i.e.,ȱ systemȱ monitoringȱ andȱ evaluation).ȱ Thereȱ isȱ noȱ universalȱ blueprintȱ forȱ buildingȱ suchȱ aȱ comprehensiveȱ assessmentȱ systemȱ becauseȱ theȱ bestȬfitȱ solutionȱ forȱ aȱ particularȱ countryȱ willȱ dependȱ uponȱitsȱobjectivesȱandȱpriorities,ȱgeopoliticalȱcontextȱandȱtraditions,ȱandȱ theȱfiscalȱandȱhumanȱresourcesȱavailable.ȱHowever,ȱgovernmentsȱaimingȱ toȱ strengthenȱ assessmentȱ practicesȱ canȱ learnȱ importantȱ generalȱ lessonsȱ fromȱ thoseȱ thatȱ areȱ furtherȱ downȱ theȱ roadȱ toȱ reform.ȱ Theȱ mainȱ purposesȱ ofȱ thisȱ caseȱ studyȱ areȱ toȱ documentȱ theȱ progressȱ madeȱ byȱ Armeniaȱ inȱ theȱ areaȱ ofȱ assessmentȱ overȱ theȱ pastȱ twoȱ decadesȱ andȱ toȱ shareȱ theȱ lessonsȱ learnedȱ withȱ otherȱ countriesȱ aimingȱ toȱ developȱ theirȱ ownȱ assessmentȱ systems.ȱ ȱ Thisȱ paperȱ considersȱ threeȱ typesȱ ofȱ assessmentȱ activitiesȱ inȱ Armenia’sȱ generalȱ educationȱ system:ȱ classroomȱ assessments,ȱ examinations,ȱ andȱ largeȬscaleȱ assessmentsȱ (bothȱ nationalȱ andȱ international).ȱ Innovationsȱ andȱ modificationsȱ toȱ preȬexistingȱ assessmentȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 1ȱ practicesȱ areȱ described,ȱ andȱ anȱ attemptȱ isȱ madeȱ toȱ identifyȱ theȱ primaryȱ driversȱ thatȱ providedȱ theȱ momentumȱ forȱ changeȱ andȱ theȱ mechanismsȱ thatȱledȱtoȱsuccessfulȱreforms.ȱ ȱ Armeniaȱ isȱ aȱ particularlyȱ interestingȱ caseȱ becauseȱ itȱ isȱ oftenȱ citedȱ asȱ anȱ exampleȱ ofȱ aȱ relativelyȱ poorȱ countryȱ thatȱ hasȱ madeȱ significantȱ andȱ rapidȱ progressȱ inȱ theȱ reformȱ ofȱ studentȱ assessmentȱ practices.ȱ (See,ȱ forȱ example,ȱ theȱ SABERȬStudentȱ Assessmentȱ Countryȱ Reportȱ forȱ Armenia,ȱ 2011.)ȱ Itsȱ achievements,ȱ however,ȱ canȱ onlyȱ beȱ fullyȱ appreciatedȱ byȱ understandingȱtheȱcontextȱinȱwhichȱreformsȱwereȱundertaken.ȱ ȱ Armeniaȱ wasȱ officiallyȱ recognizedȱ asȱ anȱ independentȱ countryȱ inȱ Septemberȱ 1991,ȱ afterȱ theȱ breakupȱ ofȱ theȱ Unionȱ ofȱ Sovietȱ Socialistȱ Republicsȱ (USSR).ȱ Itȱ isȱ aȱ relativelyȱ smallȱ countryȱ inȱ challengingȱ geographicalȱ andȱ politicalȱ environments. 1 ȱ Theȱ populationȱ isȱ justȱ 3ȱ million,ȱbut,ȱasȱaȱresultȱofȱturbulentȱeventsȱinȱtheȱearlyȱtwentiethȱcentury,ȱ thereȱ isȱ aȱ diasporaȱ ofȱ approximatelyȱ 8ȱ millionȱ ethnicȱ Armeniansȱ livingȱ outsideȱ ofȱ theȱ country.ȱ Comparedȱ withȱ otherȱ formerȱ socialistȱ republics,ȱ theȱ internalȱ populationȱ isȱ veryȱ homogenous,ȱ comprisingȱ 98ȱ percentȱ ethnicȱ Armenians.ȱ Thereȱ isȱ aȱ strongȱ senseȱ ofȱ nationalȱ identityȱ thatȱ isȱ reinforcedȱ throughȱ aȱ nationalȱ languageȱ spokenȱ onlyȱ byȱ Armeniansȱ andȱ writtenȱinȱaȱuniqueȱscript.2ȱ ȱ Theȱ country’sȱ economicȱ fortunesȱ haveȱ fluctuatedȱ dramaticallyȱ sinceȱ independence.ȱ Immediatelyȱ followingȱ theȱ transition,ȱ theȱ economyȱ wasȱ weakȱ andȱ heavilyȱ dependentȱ uponȱ remittancesȱ fromȱ theȱ diaspora.ȱ However,ȱ strategicȱ decisionsȱ byȱ theȱ Governmentȱ ofȱ theȱ Republicȱ ofȱ Armeniaȱ (GORA)ȱ generatedȱ impressiveȱ growthȱ overȱ aȱ 15Ȭyearȱ period.3ȱ Unfortunately,ȱ theȱ economyȱ wasȱ hardȱ hitȱ byȱ theȱ globalȱ financialȱ crisisȱ ofȱ 2008,ȱ withȱ aȱ 14ȱ percentȱ dropȱ inȱ Grossȱ Domesticȱ Productȱ (GDP)ȱ beingȱ recordedȱ inȱ 2009.ȱ Thereȱ areȱ nowȱ signsȱ ofȱ aȱ recoveryȱ butȱ theȱ currentȱ GDPȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 1ȱ Armeniaȱ isȱ aȱ landlockedȱ countryȱ inȱ theȱ mountainsȱ ofȱ theȱ Southȱ Caucasusȱ regionȱ sharingȱ bordersȱwithȱTurkey,ȱGeorgia,ȱIran,ȱandȱAzerbaijan.ȱItȱhasȱanȱareaȱofȱ29,743ȱkm².ȱ 2ȱ TheȱArmenianȱalphabetȱisȱuniqueȱand,ȱwithȱtheȱexceptionȱofȱaȱfewȱlaterȱadditions,ȱremainsȱ unchangedȱ fromȱ thatȱ introducedȱ byȱ theȱ theologianȱ andȱ scholarȱ Mesropȱ Mashtotsȱ aroundȱ 405ȱCommonȱEra.ȱ ȱ 3ȱ Fromȱ1998ȱuntilȱ2013,ȱArmenia’sȱGDPȱAnnualȱGrowthȱRateȱaveragedȱ8.2ȱpercent,ȱreachingȱ anȱ allȬtimeȱ highȱ ofȱ 19.1ȱ percentȱ inȱ Juneȱ ofȱ 2006ȱ (2007–2009ȱ MediumȬTermȱ Publicȱ ExpenditureȱFramework,ȱGORA).ȱ ȱ ȱ 2ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ annualȱ growthȱ rateȱ ofȱ betweenȱ 3ȱ andȱ 5ȱ percentȱ remainsȱ farȱ belowȱ theȱ doubleȬdigitȱgrowthȱofȱtheȱpreȬcrisisȱperiod.ȱ Inȱ Armenia,ȱ educationȱ hasȱ longȱ beenȱ regardedȱ asȱ theȱ mainȱ factorȱ inȱ maintainingȱtheȱnation’sȱidentity,ȱsurvival,ȱandȱprogress,ȱand,ȱasȱaȱresult,ȱ remainsȱ highlyȱ valuedȱ byȱ theȱ publicȱ andȱ aȱ priorityȱ forȱ theȱ Government.ȱ Sinceȱ independence,ȱ significantȱ reformsȱ haveȱ beenȱ implementedȱ acrossȱ allȱ aspectsȱ ofȱ theȱ educationȱ system.ȱ Someȱ vestigesȱ ofȱ theȱ Sovietȱ systemȱ haveȱbeenȱretained,ȱbutȱonlyȱthoseȱconsideredȱbeneficial.ȱTheȱstructureȱofȱ theȱ educationȱ system,ȱ includingȱ bothȱ generalȱ andȱ professionalȱ educationȱ programs,ȱ isȱ shownȱ inȱ Figureȱ 1.ȱ Generalȱ educationȱ forȱ approximatelyȱ 370,000ȱstudentsȱisȱprovidedȱbyȱ1,388ȱstateȱschoolsȱandȱ47ȱprivateȱschools.ȱ Justȱ 2ȱ percentȱ (~7,500)ȱ ofȱ theȱ cohortȱ attendȱ privateȱ schools.ȱ Inȱ addition,ȱ approximatelyȱ 36,000ȱ studyȱ inȱ Vocationalȱ Educationȱ andȱ Trainingȱ (VET)ȱ collegesȱ (99)ȱ orȱ schoolsȱ (25).ȱ Thereȱ areȱ 70ȱ Higherȱ Educationȱ Institutionsȱ (HEI),ȱ ofȱ whichȱ 22ȱ areȱ stateȬowned,ȱ 35ȱ areȱ privatelyȬowned,ȱ fiveȱ areȱ intergovernmental,ȱ andȱ eightȱ areȱ branchesȱ ofȱ foreignȱ universities. 4ȱ Approximatelyȱ103,000ȱstudentsȱareȱcurrentlyȱenrolledȱinȱHEI.ȱ Thisȱ paperȱ discussesȱ threeȱ mainȱ typesȱ ofȱ assessmentȱ activityȱ inȱ Armenia’sȱgeneralȱeducationȱsystem:ȱclassroomȱassessmentsȱ(particularlyȱ thoseȱ designedȱ toȱ yieldȱ informationȱ toȱ supportȱ teachingȱ andȱ learning),ȱ examinationsȱ (whereȱ resultsȱ areȱ usedȱ toȱ makeȱ highȬstakesȱ decisionsȱ aboutȱ individualȱ students),ȱ andȱ largeȬscaleȱ assessmentsȱ (forȱ monitoringȱ performanceȱ atȱ theȱ systemȱ level).5ȱ Forȱ eachȱ assessmentȱ type,ȱ threeȱ keyȱ factorsȱ (i.e.,ȱ ‘qualityȱ drivers’)ȱ areȱ considered.ȱ Theseȱ are:ȱ theȱ enablingȱ contextȱ inȱ whichȱ assessmentȱ activitiesȱ takeȱ place;ȱ theȱ degreeȱ toȱ whichȱ assessmentȱ activitiesȱ areȱ alignedȱ withȱ broaderȱ aspectsȱ ofȱ theȱ educationȱ system,ȱ including,ȱ forȱ example,ȱ curricularȱ objectivesȱ andȱ goals;ȱ andȱ theȱ technicalȱ qualityȱ ofȱ theȱ assessment,ȱ includingȱ itsȱ validityȱ withȱ respectȱ toȱ itsȱintendedȱpurposes.ȱ ȱ ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 4ȱ Intergovernmentalȱ HEIȱ inȱ Armeniaȱ includeȱ theȱ RussianȬArmenianȱ (Slavonic)ȱ University,ȱ theȱ Americanȱ Universityȱ ofȱ Armenia,ȱ theȱ Frenchȱ Universityȱ ofȱ Armenia,ȱ theȱ Europeanȱ RegionalȱEducationalȱAcademyȱofȱArmenia,ȱandȱtheȱFrenchȱHigherȱInstituteȱofȱEngineeringȱ inȱArmenia.ȱ 5 ȱ Thisȱ reflectsȱ theȱ generalȱ structureȱ ofȱ theȱ READ/SABERȱ Frameworkȱ forȱ Studentȱ AssessmentȱSystems,ȱasȱdescribedȱinȱClarkeȱ(2012).ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 3ȱ Figureȱ1:ȱStructureȱofȱtheȱArmenianȱEducationȱSystemȱasȱofȱ2015ȱ Doctor of Sciences Post –Graduate Education Candidate of Sciences (3 years) Researcher (3 years) Master (1-2 years) Education Bachelor (3-4 years) Higher Middle Level Vocational Education /Colleges (3 years) Ages 15 to 17 Upper Secondary Education/ Initial Vocational Education High School (3 years) (6 months to 1 year) Ages 15 to 17 Ages 15 to 16 Education (12 years) Full Secondary Middle Secondary Education (5 years) Ages 10 to 14 Primary Education (4 years) Ages 6 to 9 Pre-School Education (3 years) Ages 3 to 5 ȱ Thisȱ generalȱ introductionȱ isȱ followedȱ byȱ anȱ overviewȱ ofȱ theȱ currentȱ studentȱ assessmentȱ systemȱ inȱ Armenia.ȱ Thereafter,ȱ theȱ threeȱ typesȱ ofȱ assessmentȱ areȱ discussedȱ inȱ detail.ȱ Forȱ each,ȱ theȱ paperȱ providesȱ aȱ descriptionȱofȱcurrentȱpractices,ȱpolicies,ȱstructures,ȱandȱresources.ȱItȱalsoȱ identifiesȱandȱevaluatesȱkeyȱdriversȱandȱmechanismsȱofȱchange.ȱTheȱfinalȱ sectionȱ ofȱ theȱ paperȱ summarizesȱ importantȱ issuesȱ andȱ findings.ȱ Inȱ addition,ȱ keyȱ lessonsȱ learned—especiallyȱ thoseȱ thatȱ mayȱ beȱ ofȱinterestȱ toȱ othersȱreformingȱorȱstrengtheningȱtheirȱstudentȱassessmentȱsystems—areȱ presented.ȱ ȱ 4ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ OverviewȱofȱtheȱStudentȱAssessmentȱSystemȱ Theȱ currentȱ studentȱ assessmentȱ systemȱ inȱ Armeniaȱ isȱ comprehensive,ȱ bothȱ inȱ itsȱ coverageȱ ofȱ theȱ primaryȱ andȱ secondaryȱ phasesȱ ofȱ generalȱ educationȱ andȱ inȱ theȱ varietyȱ ofȱ assessmentȱ practicesȱ used.ȱ Theȱ systemȱ mayȱ haveȱ someȱ redundancyȱ orȱ inefficiencies,ȱ andȱ thereȱ areȱ undoubtedlyȱ areasȱwhereȱfurtherȱreformsȱareȱneeded,ȱbutȱtheȱfundamentalȱstructureȱisȱ sound,ȱ asȱ reflectedȱ inȱ theȱ generallyȬpositiveȱ findingsȱ ofȱ theȱ SABERȬStudentȱ Assessmentȱ benchmarkingȱ exerciseȱ conductedȱ inȱ 2011ȱ (WorldȱBank,ȱ2011).ȱThisȱisȱaȱfarȱcryȱfromȱtheȱsituationȱinheritedȱfromȱtheȱ Sovietȱperiod,ȱtheȱmajorȱweaknessesȱofȱwhichȱareȱdescribedȱbelow.ȱ ȱ Beforeȱ theȱ launchȱ ofȱ reforms,ȱ theȱ Ministryȱ ofȱ Educationȱ andȱ Scienceȱ (MOES)ȱ wasȱ responsibleȱ forȱ allȱ formsȱ ofȱ studentȱ assessment,ȱ includingȱ stateȱ graduationȱ examinations.ȱ Unfortunately,ȱ butȱ notȱ surprisingly,ȱ theȱ Ministryȱwasȱnotȱwellȱplacedȱtoȱmanageȱthisȱhugeȱtask.ȱFirst,ȱitȱlackedȱtheȱ necessaryȱ human,ȱ physical,ȱ andȱ financialȱ resources.ȱ Forȱ example,ȱ duringȱ examinationȱ sessions,ȱ almostȱ allȱ Ministryȱ staffȱ wereȱ involvedȱ inȱ logisticsȱ andȱ administration,ȱ meaningȱ thatȱ allȱ otherȱ ministerialȱ functionsȱ wereȱ effectivelyȱ paralyzed.ȱ Second,ȱ butȱ equallyȱ important,ȱ theȱ MOESȱ wasȱ hamperedȱ byȱ aȱ narrowȱ viewȱ ofȱ assessmentȱ asȱ ‘testing’,ȱ whichȱ wasȱ reinforcedȱbyȱlimitedȱexpertiseȱinȱthisȱspecializedȱfield.ȱ Continuousȱclassroomȱassessmentȱwasȱleftȱentirelyȱtoȱtheȱschoolȱandȱ classroomȱ levels.ȱ Teachersȱ assiduouslyȱ followedȱ administrativeȱ guidelinesȱonȱtheȱtestingȱofȱpupils,ȱbutȱfeedbackȱmechanismsȱwereȱatȱbestȱ weak.ȱ Theȱ assumptionȱ wasȱ thatȱ allȱ teachersȱ wereȱ reliableȱ assessors,ȱ butȱ theȱ truthȱ isȱ thatȱ theyȱ wereȱ poorlyȱ prepared,ȱ withȱ littleȱ orȱ noȱ trainingȱ inȱ assessmentȱtechniquesȱandȱtheirȱuses.ȱ ȱ Forȱtheȱpublic,ȱtheȱgreatestȱconcernsȱwereȱreservedȱforȱtheȱuniversityȱ entranceȱexaminationsȱconductedȱbyȱsubjectȱcommissionsȱinȱuniversities.ȱ Formally,ȱ theȱ examinationȱ papersȱ wereȱ supposedȱ toȱ beȱ basedȱ onȱ theȱ upperȬsecondaryȱ schoolȱ curriculum.ȱ Inȱ practice,ȱ however,ȱ thereȱ wasȱ aȱ significantȱgapȱbetweenȱwhatȱwasȱtaughtȱinȱschoolsȱandȱtheȱrequirementsȱ forȱ admission.ȱ Thisȱ placedȱ extraȱ pressureȱ onȱ studentsȱ preparingȱ forȱ entranceȱ examinationsȱ andȱ ledȱ mostȱ parentsȱ toȱ believeȱ thatȱ expensiveȱ privateȱ tutoringȱ wasȱ essentialȱ forȱ theȱ successȱ ofȱ theirȱ children.ȱ Inȱ addition,ȱaȱnumberȱofȱspecialȱarrangementsȱmadeȱtheȱsystemȱinequitable.ȱ Forȱ example,ȱ someȱ specializedȱ upperȬsecondaryȱ schoolsȱ reachedȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 5ȱ agreementsȱ withȱ universityȱ departmentsȱ thatȱ exemptedȱ theirȱ studentsȱ fromȱ entranceȱ examinationsȱ andȱ requiredȱ themȱ onlyȱ toȱ passȱ anȱ interview.6ȱ Similarly,ȱstudentsȱwhoȱhadȱwonȱaȱ‘GoldȱMedal’ȱorȱhadȱbeenȱ successfulȱ inȱ aȱ subjectȱ Olympiadȱ hadȱ toȱ takeȱ onlyȱ oneȱ examination,ȱ withȱ someȱ beingȱ admittedȱ simplyȱ onȱ theȱ basisȱ ofȱ anȱ interview.ȱ Inȱ short,ȱ priorȱ toȱtheȱreforms,ȱtheȱtraditionalȱsystemȱofȱuniversityȱentranceȱexaminationsȱ wasȱinefficientȱandȱunfair.ȱ Throughȱ policiesȱ adoptedȱ byȱ theȱ Government,ȱ technicalȱ developmentȱ activitiesȱ undertakenȱ throughȱ Worldȱ BankȬsupportedȱ projects,ȱ andȱ regulationsȱ andȱ guidelinesȱ enactedȱ byȱ theȱ MOES,ȱ manyȱ ofȱ theseȱ issuesȱ haveȱ beenȱ addressed,ȱ asȱ seenȱ inȱ theȱ descriptionsȱ ofȱ currentȱ assessmentȱpracticesȱwhichȱfollow.ȱ ȱ ClassroomȱAssessmentȱ Theȱ continuousȱ assessmentȱ ofȱ studentsȱ byȱ teachersȱ wasȱ aȱ significantȱ featureȱofȱtheȱSovietȱsystemȱandȱoneȱthatȱwasȱvaluedȱandȱunderstood,ȱnotȱ onlyȱbyȱteachers,ȱbutȱalsoȱbyȱstudentsȱandȱtheirȱparents.ȱAsȱinȱmanyȱotherȱ formerȱ Sovietȱ republics,ȱ Armeniaȱ hasȱ retainedȱ desirableȱ elementsȱ ofȱ thisȱ schoolȬbasedȱ system,ȱ whilstȱ adjustingȱ otherȱ aspectsȱ toȱ correspondȱ toȱ currentȱ thinkingȱ andȱ pedagogicalȱ objectives.ȱ Forȱ example,ȱ inȱ lineȱ withȱ traditionalȱ practices,ȱ teachersȱ areȱ encouragedȱ toȱ useȱ aȱ rangeȱ ofȱ familiarȱ assessmentȱtechniques,ȱtheȱmostȱcommonȱofȱwhichȱareȱ‘personalȱenquiry’ȱ (i.e.,ȱ questioningȱ andȱ oralȱ tests),ȱ ‘controlȱ works’ȱ (i.e.,ȱ writtenȱ tasksȱ setȱ specificallyȱ forȱ theȱ purposeȱ ofȱ assessment),ȱ andȱ assessedȱ homeworkȱ tasks.ȱ Teachersȱ gradeȱ allȱ formsȱ ofȱ assessmentȱ onȱ aȱ tenȬpointȱ scale,ȱ whereȱ 9ȱ toȱ 10ȱ isȱ ‘excellent’,ȱ 7ȱ toȱ 8ȱ isȱ ‘good’,ȱ 5ȱ toȱ 6ȱ isȱ ‘satisfactory’,ȱ and,ȱ 1ȱ toȱ 4ȱ isȱ ‘failing’.ȱ Studentȱ resultsȱ areȱ recordedȱ inȱ theȱ classȱ journal.ȱ Thereȱ isȱ noȱ formalȱ standardizationȱ acrossȱ schools,ȱ butȱ theȱ scoringȱ systemȱ isȱ familiarȱ toȱ allȱ andȱ scoresȱ areȱ widelyȱ usedȱ asȱ sufficientlyȱ reliableȱ indicatorsȱ ofȱ achievement.ȱInȱshort,ȱtheȱeducationalȱauthoritiesȱandȱtheȱpublicȱcontinueȱ toȱ placeȱ greatȱ trustȱ inȱ theȱ evaluationsȱ madeȱ byȱ schoolsȱ andȱ teachers.ȱ Itȱ shouldȱ beȱ noted,ȱ however,ȱthatȱ theȱ primaryȱfunctionȱofȱ theseȱ assessmentȱ activitiesȱ isȱ toȱ recordȱ studentȱ achievement;ȱ i.e.,ȱ theyȱ areȱ summativeȱ inȱ nature.ȱ ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ ȱ Forȱexample,ȱschoolsȱwithȱspecializationsȱinȱmathematicsȱandȱphysicsȱhadȱthisȱagreementȱ 6 withȱtheȱMathematicsȱandȱPhysicsȱDepartmentsȱofȱYerevanȱStateȱUniversity.ȱ ȱ 6ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ Notwithstandingȱ theȱ above,ȱ significantȱ effortsȱ haveȱ beenȱ madeȱ toȱ reȬdirectȱ classroomȱ assessmentȱ practicesȱ towardsȱ gatheringȱ informationȱ toȱ beȱ usedȱ diagnosticallyȱ inȱ supportȱ ofȱ ongoingȱ teachingȱ andȱ learning.ȱ Forȱ example,ȱ theȱ Worldȱ BankȬsupportedȱ Firstȱ Educationȱ Qualityȱ andȱ Relevanceȱ Projectȱ (EQRPȱ I)ȱ implementedȱ anȱ extensiveȱ inȬserviceȱ trainingȱ exerciseȱ toȱ introduceȱ teachersȱ toȱ theȱ useȱ ofȱ assessmentȱ techniquesȱ forȱ purelyȱ formativeȱ purposes.ȱ Thisȱ approachȱ isȱ requiredȱ byȱ theȱ governmentȬapprovedȱ Studentȱ Assessmentȱ Conceptȱ (GORA,ȱ 2005)ȱ documentȱ andȱ supportedȱ byȱ methodologicalȱ guidelinesȱ forȱ teachersȱ publishedȱ byȱ theȱ Nationalȱ Instituteȱ forȱ Educationȱ (NIE,ȱ 2013).ȱ Itȱ shouldȱ alsoȱ beȱ notedȱ thatȱ teacherȱ assessmentȱ inȱ Gradeȱ 1ȱ isȱ designatedȱ forȱ diagnosticȱandȱmonitoringȱpurposesȱonly—summativeȱassessmentȱinȱtheȱ firstȱ yearȱ ofȱ schoolingȱ isȱ explicitlyȱ prohibitedȱ byȱ theȱ Lawȱ onȱ Generalȱ Educationȱ(GORA,ȱ2009).7ȱ Itȱ wouldȱ beȱ inaccurateȱ toȱ sayȱ thatȱ allȱ teachersȱ areȱ nowȱ fullyȱ committedȱ toȱ theȱ conceptȱ ofȱ formativeȱ assessment,ȱ butȱ theȱ degreeȱ ofȱ ‘professionalȱ inertia’ȱ isȱ probablyȱ noȱ greaterȱ thanȱ thatȱ encounteredȱ inȱ manyȱ otherȱ countries,ȱ includingȱ thoseȱ withȱ farȱ greaterȱ resourcesȱ thanȱ Armenia.8ȱ ȱ Examinationsȱ Stateȱ graduationȱ examinationsȱ atȱ theȱ endsȱ ofȱ Gradesȱ 4,ȱ 9,ȱ andȱ 12ȱ areȱ aȱ significantȱ featureȱ ofȱ theȱ assessmentȱ system.ȱ Theȱ Gradeȱ 4ȱ examinationȱ isȱ intendedȱtoȱcertifyȱthatȱstudentsȱhaveȱsuccessfullyȱcompletedȱtheȱprimaryȱ phaseȱ ofȱ education,ȱ andȱ focusesȱ onȱ Armenianȱ Languageȱ andȱ Mathematics.ȱ Individualȱ schoolsȱ setȱ theirȱ ownȱ testsȱ accordingȱ toȱ guidelinesȱapprovedȱbyȱtheȱMOESȱandȱmarkȱtheirȱownȱstudents’ȱpapers.ȱ Veryȱ fewȱ studentsȱ failȱ thisȱ examination—inȱ 2012,ȱ justȱ 33ȱ studentsȱ failedȱ outȱ ofȱ 3,1254—andȱ thereȱ isȱ aȱ growingȱ awarenessȱ thatȱ thisȱ legacyȱ examinationȱnowȱservesȱlittleȱpurposeȱsinceȱtransitionȱtoȱGradeȱ5ȱis,ȱtoȱallȱ intentsȱandȱpurposes,ȱautomatic.ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 7ȱ Articleȱ16,ȱpara.ȱ3ȱstatesȱthatȱ“Examinationȱofȱknowledgeȱofȱaȱchildȱduringȱ…ȱtheȱfirstȱyearȱ ofȱgeneralȱeducationȱschoolȱshallȱbeȱprohibited.”ȱ 8ȱ Forȱ example,ȱ aȱ reportȱ intoȱ theȱ impactȱ ofȱ aȱ nationalȱ strategyȱ forȱ supportingȱ assessmentȱ forȱ learningȱ inȱ schoolsȱ inȱ Englandȱ reportedȱ thatȱ “althoughȱ teachersȱ andȱ seniorȱ leadersȱ valuedȱ theȱ trainingȱ andȱ supportȱ theyȱ hadȱ receivedȱ …ȱ thisȱ didȱ notȱ necessarilyȱ leadȱ toȱ effectiveȱ assessmentȱforȱlearningȱinȱtheirȱschools”ȱ(Ofsted,ȱ2008,ȱp.4).ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 7ȱ Theȱ Gradesȱ 9ȱ andȱ 12ȱ stateȱ graduationȱ examinationsȱ coverȱ fiveȱ subjectȱ areas:ȱ Armenianȱ Languageȱ andȱ Literature,ȱ Mathematics,ȱ Armenianȱ History,ȱ Sciences,ȱ andȱ Foreignȱ Languages.ȱ Theȱ questionȱ papersȱ areȱ preparedȱ byȱ theȱ Assessmentȱ andȱ Testingȱ Centreȱ (ATC)ȱ andȱ administeredȱ inȱ schoolsȱ byȱ anȱ examinationȱ commissionȱ establishedȱ byȱ theȱschool’sȱDirector.ȱPassȱratesȱatȱGradesȱ9ȱandȱ12ȱareȱextremelyȱhighȱ(inȱ excessȱ ofȱ 98ȱ percent).ȱ Theȱ examinationsȱ haveȱ littleȱ powerȱ ofȱ discriminationȱandȱmerelyȱserveȱtoȱindicateȱthatȱaȱstudentȱhasȱcompletedȱ theȱ relevantȱ phaseȱ ofȱ educationȱ andȱ hasȱ reachedȱ veryȱ minimalȱ levelsȱ ofȱ achievement.ȱ Successȱ isȱ recordedȱ onȱ aȱ student’sȱ schoolȱ completionȱ certificateȱ (diploma),ȱ whichȱ isȱ aȱ necessaryȱ qualificationȱ forȱ continuingȱ educationȱin,ȱforȱexample,ȱVETȱinstitutionsȱandȱalsoȱhasȱsomeȱvalueȱinȱtheȱ employmentȱmarket.ȱ Forȱ schoolȱ leaversȱ wishingȱ toȱ enterȱ higherȱ education,ȱ theȱ Unifiedȱ Examinationȱ (UE),ȱ introducedȱ inȱ 2006,ȱ servesȱ twoȱ purposes:ȱ schoolȱ graduationȱ andȱ universityȱ entrance.ȱ Asȱ such,ȱ itȱ tendsȱ toȱ dominateȱ theȱ secondaryȱ phaseȱ ofȱ educationȱ andȱ theȱ stakesȱ associatedȱ withȱ itȱ areȱ extremelyȱ high.ȱ Theȱ examinationȱ isȱ developedȱ andȱ conductedȱ underȱ theȱ auspicesȱ ofȱ theȱ ATC.ȱ Theȱ UEȱ exertsȱ considerableȱ pressuresȱ onȱ schools,ȱ teachers,ȱ students,ȱ andȱ theirȱ familiesȱ andȱ thisȱ tendsȱ toȱ distortȱ notȱ onlyȱ whatȱ isȱ taught,ȱ butȱ alsoȱ howȱ itȱ isȱ taught.ȱ Inȱ particular,ȱ manyȱ candidatesȱ receiveȱadditionalȱprivateȱtutoringȱtoȱincreaseȱtheirȱchancesȱofȱsuccessȱonȱ theȱ UE.ȱ Notwithstandingȱ anyȱ negativeȱ impactȱ thatȱ itȱ mayȱ have,ȱ theȱ UEȱ representsȱ aȱ markedȱ improvementȱ onȱ theȱ Sovietȱ systemȱ ofȱ decentralizedȱ universityȱ entranceȱ examinationsȱ whichȱ itȱ replaced.ȱ Inȱ particular,ȱ itȱ isȱ demonstrablyȱ moreȱ equitable,ȱ moreȱ transparent,ȱ andȱ lessȱ proneȱ toȱ corruption.ȱ NationalȱandȱInternationalȱLargeȬScaleȱAssessmentsȱ Classroomȱ assessmentsȱ andȱ examinationsȱ focusȱ onȱ measuringȱ theȱ achievementȱ ofȱ individualȱ studentsȱ forȱ specificȱ purposes.ȱ Theyȱ areȱ not,ȱ however,ȱ wellȱ designedȱ forȱ monitoringȱ overallȱ standardsȱ atȱ theȱ systemȱ level,ȱ orȱ forȱ investigatingȱ theȱ backgroundȱ factorsȱ thatȱ canȱ contributeȱ toȱ higherȱ levelsȱ ofȱ studentȱ achievement.ȱ Theseȱ functionsȱ areȱ betterȱ servedȱ byȱ largeȬscaleȱ assessmentsȱ conductedȱ atȱ theȱ nationalȱ andȱ internationalȱ levels.ȱSinceȱ2003,ȱArmeniaȱhasȱparticipatedȱinȱtheȱTrendsȱinȱInternationalȱ ȱ 8ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ Mathematicsȱ andȱ Scienceȱ Studyȱ (TIMSS).ȱ Itȱ isȱ currentlyȱ participatingȱ inȱ theȱ2015ȱcycle.9ȱ ȱ Moreȱ recently,ȱ inȱ orderȱ toȱ complementȱ itsȱ involvementȱ inȱ TIMSS,ȱ Armeniaȱ hasȱ designedȱ itsȱ ownȱ national,ȱ largeȬscaleȱ assessmentȱ (NLSA).ȱ Thisȱ isȱ aȱ sampleȬbasedȱ surveyȱ thatȱ adoptsȱ severalȱ ofȱ theȱ technicalȱ proceduresȱ ofȱ TIMSS,ȱ butȱ focusesȱ onȱ achievementȱ inȱ keyȱ subjectȱ areasȱ ofȱ theȱ Armenianȱ curriculum.ȱ Theȱ ATCȱ isȱ responsibleȱ forȱ allȱ aspectsȱ ofȱ theȱ nationalȱ assessmentȱ andȱ conductedȱ theȱ firstȱ cycleȱ inȱ 2010.ȱ Theȱ 2010ȱ assessmentȱ focusedȱ onȱ studentȱ achievementȱ inȱ theȱ subjectȱ areasȱ ofȱ ArmenianȱLanguageȱandȱLiterature,ȱandȱArmenianȱHistory.ȱ ȱ Policies,ȱOrganizationalȱStructures,ȱandȱResourcesȱ Keyȱ policiesȱ concerningȱ studentȱ assessmentȱ proceduresȱ areȱ setȱ outȱ inȱ governmentȱ legislation.ȱ Theȱ primaryȱ legislationȱ isȱ theȱ Lawȱ onȱ Educationȱ adoptedȱ inȱ 1999ȱ (GORA,ȱ 1999).ȱ Subsequentȱ lawsȱ regulateȱ theȱ separateȱ levelsȱandȱfunctionsȱofȱtheȱeducationȱsector.ȱMostȱrelevantȱhereȱisȱtheȱLawȱ onȱ Generalȱ Educationȱ ofȱ 2009ȱ andȱ subsequentȱ amendmentsȱ (GORA,ȱ 2009).ȱ ȱ TheȱMOESȱoverseesȱtheȱimplementationȱofȱtheȱlawȱinȱschoolsȱandȱasȱ suchȱ isȱ responsibleȱ forȱ settingȱ regulationsȱ forȱ theȱ deliveryȱ ofȱ theȱ curriculumȱ andȱ theȱ assessmentȱ ofȱ learners.ȱ Normsȱ forȱ classroomȱ assessmentsȱ andȱ theȱ organizationȱ ofȱ theȱ stateȱ graduationȱ examinationsȱ areȱ setȱ byȱ Ministerialȱ decree.ȱ Theȱ mostȱ recentȱ ofȱ theseȱ isȱ theȱ decreeȱ prescribingȱ Proceduresȱ forȱ Assessmentȱ ofȱ Studentȱ Progress,ȱ adoptedȱ inȱ 2014ȱ(MOES,ȱ2014).ȱTheȱonlyȱformȱofȱassessmentȱthatȱdoesȱnotȱfallȱdirectlyȱ underȱtheȱauspicesȱofȱtheȱMOESȱisȱtheȱUE.ȱThisȱisȱconductedȱbyȱtheȱATCȱ which,ȱsinceȱ2009,ȱhasȱbeenȱsubordinatedȱtoȱtheȱGovernmentȱCabinetȱandȱ henceȱ theȱ Primeȱ Minister.10ȱ Policiesȱ concerningȱ theȱ UEȱ areȱ setȱ outȱ inȱ theȱ prevailingȱlawsȱandȱamendmentsȱonȱGeneralȱEducation.ȱ Theȱ ATCȱ servesȱ asȱ theȱ leadȱ institutionȱ forȱ examinationsȱ andȱ largeȬscaleȱ assessments.ȱ Thereȱ areȱ 81ȱ membersȱ ofȱ staff,ȱ includingȱ 15ȱ assessmentȱ professionalsȱ inȱ theȱ Departmentȱ ofȱ Testologyȱ andȱ 12ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 9ȱ Inȱ Armenia,ȱ theȱ mainȱ dataȱ collectionȱ forȱ TIMSSȱ 2015ȱ wasȱ completedȱ inȱ Mayȱ 2015.ȱ Theȱ internationalȱreportȱisȱdueȱtoȱbeȱreleasedȱinȱDecember,ȱ2016.ȱ ȱ 10ȱ Theȱ ATCȱ wasȱ formallyȱ createdȱ onȱ Aprilȱ 14,ȱ 2004ȱ byȱ Governmentȱ Decisionȱ #586ȬNȱ asȱ aȱ separateȱ institutionȱ underȱ supervisionȱ ofȱ theȱ MOES,ȱ withȱ theȱ legalȱ statusȱ ofȱ aȱ state,ȱ nonȬprofitȱorganization.ȱItȱwasȱtransferredȱtoȱtheȱGovernmentȱCabinetȱinȱ2009.ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 9ȱ specialistsȱ inȱ theȱ Departmentȱ ofȱ ICTȱ andȱ Analysis.ȱ ATCȱ isȱ fundedȱ throughȱaȱdedicatedȱlineȱinȱtheȱStateȱbudget.ȱTheȱ2014ȱbudgetȱallocatedȱaȱ totalȱ ofȱ AMDȱ 413ȱ millionȱ (~USDȱ 1ȱ million)ȱ forȱ annualȱ costs,ȱ includingȱ staffȱ salariesȱ andȱ utilities.ȱ ATCȱ isȱ alsoȱ permittedȱ toȱ generateȱ incomeȱ throughȱ studentȱ feesȱ forȱ takingȱ theȱ UE.ȱ Inȱ 2013,ȱ thisȱ generatedȱ approximatelyȱAMDȱ36ȱmillionȱ(~USDȱ90,000)ȱinȱincome.ȱ Beyondȱ budgetaryȱ provisionȱ forȱ ATC,ȱ few,ȱ ifȱ any,ȱ fundsȱ areȱ explicitlyȱ earȬmarkedȱ forȱ studentȱ assessment.ȱ Inȱ particular,ȱ teachersȱ areȱ expectedȱ toȱ conductȱ continuousȱ assessmentȱ andȱ markȱ stateȱ graduationȱ examinationsȱ asȱ partȱ ofȱ theirȱ normalȱ duties,ȱ i.e.,ȱ withoutȱ extraȱ payment.ȱ Similarly,ȱ anyȱ costsȱ incurredȱ inȱ organizingȱ graduationȱ examsȱ areȱ metȱ fromȱschoolȱbudgets.ȱ ClassroomȱAssessmentȱ Overviewȱ Atȱ independence,ȱ Armeniaȱ inheritedȱ aȱ robustȱ systemȱ ofȱ classroomȱ assessmentȱ inȱ whichȱ teachersȱ wereȱ requiredȱ toȱ evaluateȱ andȱ recordȱ studentȱ achievementȱ regularlyȱ soȱ thatȱ progressȱ couldȱ beȱ monitoredȱ byȱ parentsȱ andȱ theȱ educationalȱ authorities.ȱ Allȱ formsȱ ofȱ assessmentȱ wereȱ gradedȱ onȱ aȱ ubiquitousȱ fiveȬpointȱ scale,ȱ whichȱ wasȱ universallyȱ understoodȱevenȱthoughȱthereȱwasȱnoȱstandardizationȱwithinȱorȱbetweenȱ schools.11ȱ Indeed,ȱteachersȱenjoyed—andȱcontinueȱtoȱenjoy—aȱgreatȱdealȱ ofȱ publicȱ trustȱ inȱ theirȱ roleȱ asȱ assessors.ȱ Thisȱ legacyȱ formedȱ aȱ firmȱ foundationȱ onȱ whichȱ toȱ buildȱ modernȱ reforms,ȱ includingȱ promotingȱ theȱ useȱofȱclassroomȱassessmentsȱforȱformativeȱpurposes.ȱ Currentȱ MOESȱ regulationsȱ (MOES,ȱ 2014)ȱ areȱ inȱ placeȱ forȱ aȱ comprehensiveȱ systemȱ ofȱ schoolȬbasedȱ assessment,ȱ whichȱ requiresȱ teachersȱtoȱassessȱtheirȱstudentsȱregularly,ȱusingȱaȱvarietyȱofȱmethods,ȱforȱ bothȱ formativeȱ andȱ summativeȱ purposesȱ andȱ toȱ recordȱ andȱ reportȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 11ȱ Onȱ thisȱ traditionalȱ scale,ȱ 1ȱ andȱ 2ȱ wereȱ consideredȱ toȱ beȱ failingȱ grades,ȱ 3ȱ indicatedȱ ‘satisfactory’,ȱ 4ȱ wasȱ ‘good’,ȱ andȱ 5ȱ wasȱ ‘excellent’.ȱ Thisȱ scaleȱ hasȱ nowȱ beenȱ replacedȱ byȱ aȱ tenȬpointȱscale,ȱbutȱtheȱgeneralȱcategoriesȱremainȱunchanged.ȱ ȱ 10ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ outcomesȱ onȱ aȱ standardȱ tenȬpointȱ scale.12ȱ Thisȱ isȱ wellȱ understoodȱ byȱ schoolsȱ and,ȱ fromȱ Gradeȱ 2ȱ onwards,ȱ studentsȱ areȱ assessedȱ throughȱ oralȱ tests,ȱ assessedȱ tasksȱ (‘controlȱ works’),ȱ andȱ occasional,ȱ writtenȱ tests.ȱ Inȱ addition,ȱ moreȱ formal,ȱ endȬofȬsemesterȱ testsȱ inȱ eachȱ curriculumȱ subjectȱ areȱ arrangedȱ byȱ schools.ȱ Gradesȱ (onȱ aȱ tenȬpointȱ scale)ȱ areȱ assiduouslyȱ recordedȱ andȱ usedȱ toȱ calculateȱ gradeȬpointȱ averagesȱ forȱ theȱ year.ȱ Inȱ practice,ȱ theȱ balanceȱ isȱ stillȱ heavilyȱ inȱ favorȱ ofȱ summativeȱ assessment,ȱ withȱ‘pure’ȱformativeȱassessmentȱfoundȱinȱaȱminorityȱofȱclassrooms.ȱPartȱ ofȱthisȱisȱdueȱtoȱtheȱfactȱthat,ȱinȱspiteȱofȱinȬserviceȱtraining,ȱmanyȱteachersȱ remainȱ unfamiliarȱ withȱ diagnosticȱ assessmentȱ techniques.ȱ Inȱ addition,ȱ theȱ pressureȱ ofȱ competitionȱ meansȱ thatȱ schoolsȱ stillȱ feelȱ compelledȱ toȱ measureȱandȱreportȱstudentȱprogressȱtoȱparentsȱinȱmoreȱtraditionalȱways.ȱ Initialȱ MOESȱ guidanceȱ toȱ teachers,ȱ designedȱ toȱ regularizeȱ andȱ controlȱ classroomȱ assessmentsȱ inȱ accordanceȱ withȱ theȱ practicesȱ developedȱunderȱEQRPȱI,ȱdrewȱmanyȱcomplaints.ȱTeachersȱargued,ȱwithȱ someȱ justification,ȱ thatȱ theȱ proceduresȱ wereȱ overlyȱ complicatedȱ andȱ thatȱ theyȱ increasedȱ theirȱ workloadȱ unacceptably.ȱ Inȱ response,ȱ theȱ newȱ guidelinesȱ offerȱ moreȱ clarityȱ andȱ greaterȱ freedomȱ forȱ teachersȱ toȱ chooseȱ howȱ often,ȱ when,ȱ andȱ howȱ theyȱ useȱ summativeȱ andȱ formativeȱ assessmentsȱ inȱ theirȱ classes. 13 ȱ Theyȱ alsoȱ provideȱ clearȱ andȱ simpleȱ guidanceȱ onȱ theȱ aggregationȱ ofȱ scoresȱ fromȱ summativeȱ assessmentsȱ toȱ produceȱannualȱgrades.ȱ Teachersȱareȱfurtherȱsupportedȱbyȱaȱpracticalȱguideȱproducedȱbyȱtheȱ NIEȱ (2013).ȱ Inȱ addition,ȱ furtherȱ initiativesȱ areȱ beingȱ introducedȱ toȱ helpȱ teachersȱ overcomeȱ theirȱ lackȱ ofȱ confidenceȱ inȱ applyingȱ newȱ classroomȱ assessmentȱ practices.ȱ Theseȱ areȱ describedȱ underȱ Mechanismsȱ ofȱ Changeȱ below.ȱ CurrentȱPolicies,ȱStructures,ȱandȱResourcesȱforȱClassroomȱAssessmentȱ GORAȱ andȱ theȱ MOESȱ haveȱ setȱ outȱ clearȱ policiesȱ concerningȱ theȱ continuousȱ assessmentȱ ofȱ studentsȱ byȱ teachers.ȱ Buildingȱ onȱ aȱ Ministerialȱ Decree,ȱ ‘Conceptȱ ofȱ Students’ȱ Continuousȱ Assessment:ȱ Generalȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 12ȱ Theȱ MOESȱ issuedȱ newȱ guidelinesȱ onȱ classroomȱ assessmentȱ inȱ 2014ȱ toȱ replaceȱ earlierȱ guidelinesȱthatȱhadȱdrawnȱvociferousȱobjectionsȱfromȱteachersȱasȱbeingȱtooȱcomplicatedȱandȱ addingȱtoȱtheirȱworkȱburden.ȱ 13ȱ Theȱ guidelinesȱ doȱ setȱ minimumȱ numbersȱ forȱ eachȱ typeȱ ofȱ assessmentȱ toȱ ensureȱ thatȱ teachersȱuseȱanȱappropriateȱvarietyȱofȱassessmentȱtechniques.ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 11ȱ Statement,ȱBasicȱApproaches’ȱ(MOES,ȱ2008),ȱcurrentȱrequirementsȱareȱsetȱ outȱ inȱ theȱ MOESȱ guidelines,ȱ ‘Proceduresȱ forȱ assessmentȱ ofȱ studentȱ progress’ȱ (MOES,ȱ 2014).ȱ Theȱ MOES,ȱ inȱ closeȱ collaborationȱ withȱ itsȱ schools,ȱ monitorsȱ theȱ implementationȱ ofȱ classroomȱ assessmentȱ policies.ȱ AdditionalȱprofessionalȱsupportȱisȱofferedȱthroughȱtheȱNIE.ȱ Fewȱ resourcesȱ areȱ explicitlyȱ allocatedȱ toȱ classroomȱassessmentȱsinceȱ itȱ isȱ seenȱ asȱ anȱ integralȱ partȱ ofȱ theȱ teaching/learningȱ processȱ which,ȱ inȱ turn,ȱisȱfundedȱthroughȱschoolȱbudgets.ȱ ȱ MechanismsȱofȱChangeȱ EQRPȱ Iȱ hadȱ aȱ broadȱ scope,ȱ includingȱ theȱ developmentȱ ofȱ newȱ curriculaȱ andȱ standardsȱ andȱ harnessingȱ theȱ positiveȱ backwashȱ effectȱ associatedȱ withȱ highȬqualityȱ assessmentȱ inȱ schools.ȱ Theȱ projectȱ allowedȱ theȱ MOESȱ toȱ launch,ȱ inȱ 2005,ȱ aȱ seriesȱ ofȱ initiativesȱ toȱ developȱ aȱ systemȱ ofȱ continuous,ȱ formativeȱ assessmentȱ inȱ classrooms,ȱ withȱ theȱ explicitȱ objectiveȱ ofȱ encouragingȱ schoolsȱ andȱ teachersȱ toȱ useȱ assessmentȱ resultsȱ forȱ diagnosticȱ purposes.ȱ Theȱ definingȱ conceptȱ wasȱ producedȱ byȱ aȱ specialistȱ workingȱ groupȱ andȱ thenȱ suggestedȱ methodsȱ wereȱ validatedȱ throughȱ extensiveȱ consultationȱ andȱ pilotingȱ (MOES,ȱ 2008).ȱ Finally,ȱ anȱ experimental,ȱ threeȬyearȱ programȱ wasȱ approvedȱ byȱ theȱ MOESȱ andȱ implementedȱ inȱ allȱ schools.14ȱ Partȱ ofȱ theȱ implementationȱ strategyȱ wasȱ toȱ provideȱ schoolsȱ withȱ modelȱ testsȱ inȱ aȱ wideȱ rangeȱ ofȱ subjects,ȱ basedȱ onȱ newȱ curriculaȱ andȱ standardsȱ developedȱ underȱ EQRPȱ I.ȱ Modelȱ testsȱ andȱ guidanceȱ forȱ theirȱ useȱ wereȱ distributedȱ toȱ allȱ schoolsȱ “soȱ thatȱ teachersȱ couldȱlearnȱnewȱquestioningȱtechniquesȱthatȱelicitȱbetterȱinformationȱandȱ leadȱ toȱ theȱ identificationȱ ofȱ anyȱ potentialȱ ‘gap’ȱ betweenȱ whatȱ hasȱ beenȱ taughtȱandȱwhatȱhasȱbeenȱlearned”ȱ(CenterȱforȱEducationȱProjectsȱ[CFEP]ȱ 2009,ȱp.24).ȱ Implementationȱ wasȱ supportedȱ byȱ aȱ subȬcomponentȱ ofȱ EQRPȱ Iȱ designedȱtoȱdevelopȱaȱshort,ȱmodularȱprogramȱforȱtheȱinȬserviceȱtrainingȱ ofȱ teachersȱ inȱ theȱ principlesȱ andȱ practicesȱ ofȱ assessment.ȱ Aȱ workingȱ groupȱofȱcentralȱtrainersȱworkedȱwithȱinternationalȱconsultantsȱtoȱdesignȱ theȱmodules,ȱtheȱmaterials,ȱandȱaȱtrainingȱprogramȱforȱlocalȱtrainersȱwho,ȱ inȱ turn,ȱ wereȱ toȱ trainȱ practicingȱ teachersȱ throughoutȱ theȱ country.ȱ Whilstȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ ȱ “Conceptȱ ofȱ Students’ȱ Continuousȱ Assessment:ȱ Generalȱ Statement,ȱ Basicȱ Approaches,”ȱ 14 approvedȱbyȱMinisterialȱDecreeȱN683ȬA/QȱofȱOctoberȱ2008.ȱ ȱ 12ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ thisȱ roundȱ ofȱ inȬserviceȱ trainingȱ wasȱ generallyȱ wellȬreceived,ȱ itsȱ longȬtermȱimpactȱisȱdifficultȱtoȱjudge.ȱMoreȱimportantly,ȱnewlyȬqualifiedȱ teachersȱjoiningȱtheȱprofessionȱdoȱnotȱhaveȱtheȱbenefitȱofȱhavingȱreceivedȱ intensiveȱ trainingȱ inȱ classroomȱ assessmentȱ methods.ȱ Therefore,ȱ MOES,ȱ withȱ theȱ supportȱ ofȱ theȱ Russiaȱ Educationȱ Aidȱ forȱ Developmentȱ (READ)ȱ Trustȱ Fund,ȱ isȱ currentlyȱ workingȱ onȱ newȱ mechanismsȱ toȱ buildȱ uponȱ theȱ progressȱmadeȱthusȱfar.ȱ Theȱfirstȱinitiativeȱisȱaȱsmall,ȱbutȱhighlyȱsignificant,ȱpartȱofȱaȱfarȱmoreȱ fundamentalȱ changeȱ relatedȱ toȱ theȱ recentȱ introductionȱ ofȱ aȱ wellȬdefinedȱ careerȱ structureȱ forȱ teachers.ȱ Thisȱ givesȱ teachersȱ theȱ opportunityȱ toȱ improveȱ theirȱ professionalȱ status—andȱ theirȱ salaries—byȱ progressingȱ throughȱaȱseriesȱ ofȱ fourȱlevelsȱ orȱ ranks.ȱProgressionȱ isȱ dependentȱuponȱaȱ numberȱofȱfactors,ȱincludingȱyearsȱofȱserviceȱandȱparticipationȱinȱaȱformalȱ systemȱ ofȱ Continuousȱ Professionalȱ Developmentȱ (CPD).ȱ Theȱ CPDȱ elementȱ requiresȱ teachersȱ toȱ takeȱ atȱ leastȱ 80ȱ hoursȱ ofȱ trainingȱ overȱ aȱ fiveȬyearȱ period.ȱ Theȱ fieldsȱ inȱ whichȱ trainingȱ isȱ requiredȱ includeȱ assessmentȱ andȱ so,ȱ throughȱ thisȱ mechanism,ȱ teachersȱ willȱ beȱ ableȱ toȱ accessȱ inȬserviceȱ trainingȱ fromȱ recognizedȱ providers,ȱ theȱ mostȱ predominantȱbeingȱtheȱNIE.ȱ Theȱ secondȱ initiativeȱ isȱ theȱ developmentȱ ofȱ aȱ preȬserviceȱ courseȱ forȱ teachersȱ toȱ ensureȱ thatȱ newȱ entrantsȱ toȱ theȱ professionȱ haveȱ aȱ soundȱ understandingȱ ofȱ assessmentȱ principlesȱ andȱ practices.ȱ Theȱ READȱ Trustȱ Fundȱ isȱ supportingȱ theȱ implementationȱ ofȱ thisȱ initiativeȱ byȱ providingȱ technicalȱ assistanceȱ toȱ theȱ sevenȱ HEIȱ thatȱ prepareȱ teachersȱ soȱ thatȱ anȱ appropriatelyȱ comprehensiveȱ andȱ demandingȱ courseȱ canȱ beȱ developed.ȱ Theȱ courseȱ willȱ likelyȱ placeȱ anȱ emphasisȱ onȱ modernȱ approachesȱ toȱ formativeȱassessmentȱandȱitsȱuseȱinȱtheȱclassroomȱ(Boyle,ȱ2014).ȱ Theȱ thirdȱ initiative,ȱ whichȱ shouldȱ inȱ theȱ longerȬtermȱ strengthenȱ theȱ professionalȱ assessmentȱ communityȱ inȱ Armenia,ȱ isȱ theȱ developmentȱ ofȱ aȱ twoȬyearȱMaster’sȱcourseȱinȱAssessment,ȱMeasurement,ȱandȱTesting.ȱThisȱ isȱ inȱ theȱ earlyȱ stagesȱ ofȱ developmentȱ andȱ isȱ beingȱ supportedȱ withȱ technicalȱ assistanceȱ providedȱ byȱ theȱ READȱ Trustȱ Fund.ȱ Classroomȱ assessmentȱmethodsȱwillȱlikelyȱbeȱoneȱofȱtheȱoptionsȱavailableȱtoȱstudentsȱ forȱinȬdepthȱstudyȱandȱresearchȱ(Hawker,ȱ2013).ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 13ȱ DriversȱofȱChangeȱ Atȱ theȱ timeȱ whenȱ EQRPȱ Iȱ wasȱ beingȱ prepared,ȱ thereȱ wasȱ growingȱ awarenessȱ inȱ theȱ internationalȱ educationȱ communityȱ thatȱ assessmentȱ forȱ learningȱ (formativeȱ assessment)ȱ hadȱ theȱ potentialȱ toȱ deliverȱ improvedȱ educationalȱoutcomesȱforȱrelativelyȱlittleȱcost.ȱInȱparticular,ȱtheȱideasȱthatȱ Blackȱ andȱ Wiliamȱ (1990)ȱ describedȱ inȱ theirȱ seminalȱ work,ȱ ȈInsideȱ theȱ Blackȱ Box:ȱ Raisingȱ Standardsȱ throughȱ Classroomȱ Assessment,”ȱ wereȱ increasinglyȱ beingȱ accepted.ȱ Inȱ Armenia,ȱ itȱ wasȱ recognizedȱ thatȱ theȱ conditionsȱ wereȱ rightȱ forȱ includingȱ formativeȱ assessmentȱ inȱ reformȱ activitiesȱbecauseȱtheȱcountryȱhadȱmaintainedȱfromȱSovietȱtimesȱaȱrespectȱ forȱ theȱ teacherȇsȱ roleȱ inȱ evaluatingȱ studentȱ progress.ȱ Theȱ existenceȱ ofȱ aȱ professionalȱ teachingȱ forceȱ withȱ aȱ traditionȱ ofȱ complianceȱ withȱ MOESȱ requirementsȱprovidedȱanȱidealȱenablingȱcontextȱforȱchange.ȱ Onceȱ includedȱ inȱ theȱ project,ȱ programȱ fundingȱ andȱ theȱ activeȱ managementȱ ofȱ theȱ projectȱ implementationȱ unitȱ (Centerȱ forȱ Educationȱ Projectsȱ[CFEP])ȱservedȱasȱadditionalȱdrivers.ȱTheȱstaffȱandȱconsultantsȱofȱ theȱ CFEPȱ developedȱ excellentȱ workingȱ relationshipsȱ withȱ membersȱ ofȱ specialistȱ workingȱ groups.ȱ Thisȱ wasȱ particularlyȱ importantȱ inȱ theȱ developmentȱ andȱ implementationȱ ofȱ theȱ inȬserviceȱ trainingȱ programȱ forȱ teachers.ȱ Anotherȱ importantȱ driverȱ ofȱ changeȱ wasȱ theȱ emergenceȱ ofȱ aȱ newȱ assessmentȱ communityȱ asȱ aȱ directȱ resultȱ ofȱ activitiesȱ conductedȱ underȱ EQRPȱI.ȱForȱexample,ȱmanyȱsubjectȱspecialistsȱwhoȱhadȱbeenȱinvolvedȱinȱ theȱ developmentȱ ofȱ modelȱ questionȱ papersȱ forȱ theȱ new,ȱ highȬstakesȱ UEȱ continuedȱ theirȱ involvementȱ asȱ membersȱ ofȱ theȱ workingȱ groupȱ thatȱ developedȱ theȱ trainingȱ programȱ forȱ classroomȱ assessment.ȱ Thisȱ continuityȱ ofȱ engagementȱ allowedȱ themȱ toȱ developȱ aȱ deeperȱ understandingȱ ofȱ theȱ fundamentalȱ principlesȱ thatȱ underpinȱ goodȱ assessmentȱpractices.15ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ ȱ TheȱcurrentȱDirectorȱofȱtheȱATCȱwasȱtrainedȱinȱtheȱprinciplesȱofȱassessmentȱasȱaȱmemberȱ 15 ofȱanȱEQRPȱIȱsubjectȱworkingȱgroup.ȱ ȱ 14ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ Examinationsȱ Overviewȱ Withinȱ theȱ generalȱ educationȱ system,ȱ studentsȱ encounterȱ twoȱ typesȱ ofȱ formalȱexamination:ȱstateȱgraduationȱexaminationsȱatȱtheȱendsȱofȱGradesȱ 4,ȱ 9,ȱ andȱ 12;ȱ andȱ theȱ UE,ȱ whichȱ sitsȱ atȱ theȱ interfaceȱ betweenȱ secondaryȱ andȱtertiaryȱeducation.ȱ Theȱgeneralȱsystemȱofȱstateȱgraduationȱexaminationsȱisȱaȱlegacyȱfromȱ Sovietȱtimes,ȱandȱtheȱmainȱpurposeȱisȱtoȱcertifyȱtheȱsuccessfulȱcompletionȱ ofȱ primary,ȱ middleȬsecondary,ȱ andȱ upperȬsecondaryȱ schooling.ȱ However,ȱ theȱ overallȱ passȱ ratesȱ forȱ theseȱ examinationsȱ (afterȱ permittedȱ retakesȱareȱtakenȱintoȱaccount)ȱapproachȱ100ȱpercent.ȱAsȱaȱresult,ȱtheyȱdoȱ notȱ presentȱ aȱ majorȱ hurdleȱ toȱ studentsȱ inȱ continuingȱ theirȱ educationȱ andȱ henceȱ theȱ stakesȱ areȱ relativelyȱ low.ȱ Indeed,ȱ someȱ haveȱ arguedȱ thatȱ theȱ Gradeȱ 4ȱ graduationȱ examinationȱ inȱ particularȱ isȱ nowȱ redundant.ȱ However,ȱothersȱholdȱthatȱtheȱestablishedȱpatternȱofȱformalȱexaminationsȱ doesȱ serveȱ toȱ punctuateȱ theȱ teachingȱ andȱ learningȱ processȱ forȱ students,ȱ andȱ theȱ Gradesȱ 9ȱ andȱ 12ȱ graduationȱ examsȱ continueȱ toȱ haveȱ some,ȱalbeitȱ limited,ȱconcreteȱvalue.ȱ ȱ Byȱ wayȱ ofȱ contrast,ȱ theȱ stakesȱ associatedȱ withȱ theȱ UEȱ takenȱ atȱ theȱ endȱ ofȱ Gradeȱ 12ȱ areȱ extremelyȱ high.ȱ Asȱ aȱ result,ȱ theȱ contentȱ ofȱ theȱ examinationȱ influences,ȱ andȱ someȱ wouldȱ sayȱ distorts,ȱ notȱ onlyȱ whatȱ isȱ taught,ȱ butȱ alsoȱ howȱ itȱ isȱ learntȱ (READ,ȱ 2012).ȱ Theȱ examination,ȱ itsȱ impactȱonȱteachingȱandȱlearning,ȱandȱplansȱforȱitsȱfutureȱdevelopmentȱareȱ discussedȱfurtherȱbelow.ȱ Stateȱgraduationȱexaminationsȱ Theȱ generalȱ systemȱ ofȱ stateȱ graduationȱ examinationsȱ wasȱ inheritedȱ fromȱ Sovietȱ times.ȱ However,ȱ significantȱ changesȱ wereȱ madeȱ toȱ theȱ designȱ andȱ deliveryȱofȱtheȱexaminationsȱduringȱtheȱperiodȱofȱreform.ȱFirst,ȱtheȱgradeȱ levelsȱ wereȱ adjustedȱ toȱ matchȱ theȱ longerȱ 12Ȭyearȱ periodȱ ofȱ generalȱ education,ȱ andȱ theȱ gradingȱ scalesȱ wereȱ changedȱ toȱ matchȱ theȱ 10Ȭȱ andȱ 20Ȭpointȱ scalesȱ introducedȱ underȱ theȱ newȱ Conceptȱ ofȱ Assessment,ȱ adoptedȱinȱ 2005ȱ(GORA,ȱ2005).ȱSecond,ȱ theȱ content,ȱformat,ȱandȱ modeȱofȱ deliveryȱ ofȱ theȱ testsȱ wereȱ changed.ȱ Forȱ example,ȱ inȱ theȱ Sovietȱ period,ȱ examinationȱ tasksȱ wereȱ preparedȱ byȱ theȱ NIEȱ withȱ questionsȱ andȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 15ȱ problemsȱ selectedȱ fromȱ textbooks.ȱ Then,ȱ onȱ theȱ specifiedȱ dayȱ andȱ time,ȱ theseȱ wereȱ transmittedȱ byȱ radio.ȱ Mostȱ ofȱ theȱ itemsȱ onȱ theȱ tests,ȱ withȱ theȱ exceptionȱofȱthoseȱforȱArmenianȱandȱforeignȱlanguages,ȱwereȱopenȬendedȱ andȱ scoredȱ subjectivelyȱ byȱ teachersȱ (Bethellȱ andȱ Harutyunyan,ȱ 2008).ȱ Now,ȱ theȱ testsȱ containȱ onlyȱ itemsȱ thatȱ canȱ beȱ scoredȱ objectively,ȱ i.e.,ȱ multipleȬchoiceȱandȱshortȬanswerȱtypes.ȱ Stateȱ graduationȱ examinationsȱ focusȱ onȱ theȱ keyȱ subjectsȱ ofȱ theȱ schoolȱprogramȱandȱtestȱpapersȱareȱcloselyȱalignedȱwithȱtheȱcontentȱofȱtheȱ nationalȱ curriculumȱ andȱ standards.ȱ Boxȱ 1ȱ summarizesȱ theȱ mainȱ characteristicsȱofȱtheȱthreeȱexaminations.ȱItȱcanȱbeȱseenȱthatȱresponsibilityȱ forȱ testȱ administration,ȱ theȱ markingȱ ofȱ students’ȱ work,ȱ andȱ theȱ issuanceȱ ofȱresultsȱisȱeffectivelyȱdelegatedȱtoȱschools.ȱStepsȱareȱtakenȱtoȱensureȱthatȱ schoolsȱfollowȱkeyȱadministrativeȱguidelines,ȱtestȱpapersȱareȱkeptȱsecure,ȱ andȱ markingȱ isȱ fair.ȱ Indeed,ȱ schoolsȱ takeȱ theȱ examinationsȱ seriouslyȱ andȱ theȱ resultsȱ are,ȱ inȱ general,ȱ trustedȱ byȱ students,ȱ parents,ȱ andȱ theȱ widerȱ public.ȱ ȱ Aȱ significantȱ changeȱ inȱ theȱ administrationȱ ofȱ theȱ Gradesȱ 9ȱ andȱ 12ȱ examinationsȱ wasȱ madeȱ possibleȱ byȱ theȱ establishmentȱ ofȱ theȱ ATCȱ underȱ EQRPȱI.ȱOriginally,ȱtheȱATC’sȱpriorityȱwasȱtheȱdesignȱandȱconductȱofȱtheȱ UE.ȱ However,ȱ onceȱ itȱ wasȱ operational,ȱ theȱ Ministryȱ wasȱ ableȱ toȱ switchȱ responsibilityȱforȱtheȱproductionȱofȱtheȱGradesȱ9ȱandȱ12ȱtestȱpapersȱtoȱtheȱ ATC,ȱ therebyȱ standardizingȱ testsȱ acrossȱ schools.ȱ Inȱ theȱ past,ȱ littleȱ quantitativeȱ informationȱ wasȱ gatheredȱ (exceptȱ forȱ schoolȱ failureȱ rates)ȱ andȱnoȱsecondaryȱanalysisȱwasȱcarriedȱout.ȱHowever,ȱinȱ2013,ȱforȱtheȱfirstȱ time,ȱ ATCȱ analyzedȱ theȱ resultsȱ ofȱ theȱ graduationȱ examinationsȱ forȱ Gradesȱ 9ȱ andȱ 12ȱ andȱ publishedȱ reportsȱ onȱ itsȱ findings.ȱ Thisȱ isȱ aȱ positiveȱ step,ȱ showingȱ ATC’sȱ desireȱ toȱ strengthenȱ theȱ useȱ ofȱ assessmentȬrelatedȱ dataȱandȱimproveȱfeedbackȱsystems.ȱ ȱ ȱ ȱ 16ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ Boxȱ1:ȱOverviewȱofȱtheȱGradesȱ4,ȱ9,ȱandȱ12ȱstateȱgraduationȱexaminationsȱ Grade 4 State Graduation Examination Subjects Two subjects: Armenian Language and Mathematics (90 minutes each). An additional Physical Training examination is also arranged. Organization Guidelines for the examinations are set by MOES and the ATC provides sample/model examination papers. The school Director is responsible for implementation. The school’s subject specialists set five versions of the tests. The five versions are distributed spirally to prevent copying. Teachers mark their students’ work and report results on a 10-point scale: 10/9 = ’excellent’, 8/7 = ’good’, 6/5 = ’satisfactory’, and 4-1 = ’fail’. Students who fail are allowed to retake the examination. Notes After permitted retakes, the pass rate in 2011 was 99.9 percent—only 33 students failed out of 31,254. Grade 9 State Graduation Examination Subjects Five subjects: Armenian Language and Literature, Mathematics, Armenian History, a Science (Physics, Chemistry, Biology, or Geography), and Foreign Language (typically Russian, English, French, or German) (90 minutes each). An additional Physical Training examination is also arranged. Organization Test papers and marking criteria are set by the ATC. Tests are delivered to schools just two hours before the examinations are due to start. All other processes are conducted under the auspices of an examination commission composed of selected teachers and headed by the school's Director. Students’ work is marked by the members of the commission, with results reported on 20-point scale. Notes Pass rates are close to 100 percent. (In 2011/2012, the pass rate was 99.6 percent, with about 130 students failing from a cohort of more than 38,000.) The Grade 9 diploma has some value in that it is necessary for continuing education in high schools or in VET institutions. For those leaving school, the diploma has some value in the employment market. Grade 12 State Graduation Examination Subjects The subjects tested are the same as for the Grade 9 examination: Armenian Language and Literature, Mathematics, Armenian History, a Science (all 120 minutes each), and a Foreign Language (90 minutes). A Physical Training examination is also arranged. Organization General arrangements as for the Grade 9 examination: ATC is responsible for delivering examination papers to schools in advance and the conduct of the examination and the marking/grading process fall under the auspices of an examination commission established and led by the school's Director. Notes Typical pass rates for this examination (after retakes) are in excess of 99 percent. (The pass rate in 2012 was 99.15 percent—298 failed out of 35,244.) The Grade 12 diploma is important in that it is necessary for continuing education at the tertiary level in VET and HEI. For those leaving school, the diploma has some value in the employment market. ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 17ȱ UnifiedȱExaminationȱ ForȱstudentsȱwishingȱtoȱapplyȱforȱfurtherȱstudiesȱinȱHEI,ȱtheȱGradeȱ12ȱUEȱ servesȱ asȱ aȱ schoolȱ graduationȱ testȱ and,ȱ mostȱ importantly,ȱ theȱ universityȱ selectionȱ examination.ȱ Theȱ latterȱ roleȱ ensuresȱ thatȱ itȱ dominatesȱ theȱ secondaryȱ educationȱ system.ȱ Bethellȱ andȱ Hartyunyanȱ (2008)ȱ describeȱ theȱ situationȱthus:ȱ ȱ Throughoutȱ theȱ world,ȱ examinationsȱ usedȱ toȱ selectȱ applicantsȱ forȱ institutionsȱ ofȱ higherȱ educationȱ areȱ saidȱ toȱ haveȱ ‘highȱ stakes.’ȱ However,ȱ nowhereȱ isȱ thisȱ moreȱ trueȱ thanȱ inȱ Armenia.ȱ Firstly,ȱ asȱ inȱ otherȱ countries,ȱ placesȱ onȱ prestigiousȱ coursesȱ areȱ prizedȱ forȱ theȱ economicȱ andȱ socialȱ advantagesȱ theyȱ bring.ȱ Secondly,ȱ Armenianȱ studentsȱ whoȱ doȱ exceptionallyȱ wellȱ inȱ selectionȱ testsȱ areȱ eligibleȱ forȱ freeȱ tuitionȱ atȱ university.ȱ Thoseȱ whoȱ fallȱ belowȱ theȱ thresholdȱ scoreȱ forȱ stateȬfundedȱ placesȱ mayȱ stillȱ goȱ onȱ toȱ study,ȱ butȱ onlyȱ ifȱ theyȱ canȱ affordȱ toȱ fundȱ themselves.ȱ Theȱ thirdȱ factor,ȱ andȱ byȱ farȱ theȱ mostȱ importantȱ forȱ youngȱ men,ȱ isȱ thatȱ thoseȱ whoȱ gainȱ (stateȬfunded)ȱ placesȱ atȱ universityȱ mayȱ deferȱ theirȱcompulsoryȱmilitaryȱserviceȱ(BethellȱandȱHartyunyan,ȱ2008,ȱp.112).16ȱ Theȱ currentȱ system,ȱ inȱ placeȱ sinceȱ 2006,ȱ wasȱ specificallyȱ designedȱ andȱimplementedȱunderȱEQRPȱIȱinȱorderȱtoȱreplaceȱtheȱtraditionalȱSovietȱ systemȱ wherebyȱ studentsȱ hadȱ toȱ applyȱ to,ȱ andȱ meetȱ theȱ entryȱ requirementsȱ of,ȱ individualȱ universitiesȱ (Bethellȱ andȱ Zabulionis,ȱ 2012).ȱ Theȱ mainȱ aimsȱ ofȱ theȱ reformȱ wereȱ toȱ increaseȱ fairnessȱ andȱ transparencyȱ withinȱ theȱ selectionȱ processȱ andȱ toȱ combatȱ widespreadȱ perceptionȱ ofȱ corruptionȱ andȱ malfeasanceȱ inȱ theȱ oldȱ system.ȱ Thisȱ requiredȱ theȱ establishmentȱ ofȱ anȱ organizationȱ withȱ theȱ capacityȱ andȱ integrityȱ necessaryȱ toȱ conductȱ aȱ highȬstakesȱ selectionȱ examinationȱ toȱ theȱ highestȱ standardsȱforȱtheȱwholeȱcountry.ȱAsȱaȱresult,ȱtheȱATCȱisȱnowȱresponsibleȱ forȱ allȱ aspectsȱ ofȱ theȱ UE.ȱ Thisȱ includesȱ notȱ onlyȱ testȱ productionȱ andȱ scoring,ȱ butȱ alsoȱ theȱ administrationȱ andȱ logisticsȱ associatedȱ withȱ conductingȱaȱhighȬstakesȱtestȱunderȱsecureȱconditions.ȱ ȱ Inȱ orderȱ toȱ fulfilȱ twoȱ distinctȱ purposes—schoolȱ graduationȱ andȱ selection—theȱ UEȱ hasȱ twoȱ tiers.ȱ Studentsȱ whoȱ doȱ notȱ wishȱ toȱ useȱ theirȱ resultsȱforȱaȱparticularȱsubjectȱinȱtheirȱuniversityȱapplicationȱoptȱforȱlevelȱ A,ȱwhichȱexaminesȱtheȱfullȱsubjectȱcurriculum,ȱbutȱonlyȱincludesȱtasksȱupȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ ȱ MaleȱapplicantsȱwhoȱgainȱstateȬfundedȱplacesȱmayȱdeferȱnationalȱserviceȱuntilȱafterȱtheirȱ 16 Bachelorȱstudies.ȱThoseȱwhoȱgainȱfeeȬpayingȱplacesȱhaveȱtoȱenterȱtheȱmilitaryȱimmediately,ȱ butȱtheirȱplacesȱareȱheldȱuntilȱtheyȱcompleteȱtheirȱtwoȱyearsȱofȱservice.ȱ ȱ ȱ 18ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ toȱaverageȱlevelsȱofȱdifficulty.ȱStudentsȱwishingȱtoȱuseȱaȱparticularȱsubjectȱ inȱ theirȱ universityȱ entranceȱ applicationȱ takeȱ anȱ examinationȱ incorporatingȱ levelȱ Aȱ andȱ theȱ moreȬdemandingȱ levelȱ B.ȱ Applicantsȱ takeȱ theȱ UEȱ inȱ specialȱ examinationȱ centersȱ establishedȱ byȱ ATC. 17 ȱ Thisȱ centralizationȱ ofȱ testȱ administrationȱ allowsȱ theȱ examinationsȱ toȱ beȱ conductedȱinȱaȱhighlyȬformal,ȱcontrolledȱenvironment.ȱ Forȱ graduationȱ purposes,ȱ studentsȱ takeȱ examinationsȱ inȱ fourȱ compulsoryȱsubjectsȱ(ArmenianȱLanguage,ȱMathematics,ȱArmenianȱHistory,ȱ andȱ oneȱ Foreignȱ Language)ȱ andȱ twoȱ electiveȱ subjectsȱ fromȱ aȱ listȱ approvedȱ byȱ theȱ MOES.ȱ However,ȱ studentsȱ wishingȱ toȱ enterȱ universityȱ alsoȱ haveȱ toȱ meetȱ admissionsȱ criteriaȱ byȱ takingȱ examinationsȱ inȱ theȱ subjectsȱ requiredȱ byȱ theȱ HEIȱ coursesȱ toȱ whichȱ theyȱ areȱ applying.ȱ Applicantsȱ canȱ applyȱ toȱ upȱ toȱ tenȱ differentȱ universities.ȱ Someȱ specialtiesȱ andȱ professionsȱ alsoȱ requireȱ applicantsȱ toȱ takeȱ additional,ȱ specializedȱ examinationsȱ separateȱ fromȱ theȱ UE.ȱByȱstudyingȱtheȱlistȱofȱexaminationȱsubjectsȱrequiredȱbyȱtheȱuniversitiesȱ andȱdepartmentsȱtoȱwhichȱtheyȱhaveȱapplied,ȱanȱapplicantȱcanȱascertainȱtheȱ totalȱnumberȱofȱexaminationsȱthatȱheȱorȱsheȱhasȱtoȱpassȱunderȱtheȱUE.ȱThereȱ isȱ noȱ limitȱ toȱ theȱ numberȱ ofȱ examinationsȱ thatȱ aȱ studentȱ canȱ takeȱ fromȱ theȱ governmentȬapprovedȱ listȱ ofȱ subjects.ȱ However,ȱ candidatesȱ areȱ requiredȱ toȱpayȱaȱfeeȱofȱAMDȱ1500ȱ(~USDȱ3.60)ȱforȱeachȱsubject.18ȱ ȱ Forȱ graduationȱ purposes,ȱ aȱ candidate’sȱ rawȱ scoreȱ onȱ theȱ levelȱ Aȱ examinationȱ isȱ transformedȱ toȱ aȱ gradeȱ onȱ theȱ 1–20ȱ scale,ȱ whereȱ 7.5ȱ isȱ theȱ minimumȱpassingȱgrade.ȱForȱuniversityȱselectionȱpurposes,ȱaȱcandidate’sȱ totalȱrawȱscoreȱonȱlevelsȱAȱandȱBȱisȱconvertedȱtoȱaȱscoreȱonȱtheȱ20Ȭpointȱscaleȱ byȱaȱsimpleȱlinearȱtransformation.ȱHereȱtoo,ȱtheȱminimumȱpassingȱgradeȱforȱ allȱ subjectsȱ isȱ setȱ atȱ 7.5.ȱ Thisȱ makesȱ theȱ procedureȱ bothȱ transparentȱ andȱ capableȱ ofȱ beingȱ explainedȱ toȱ aȱ highlyȬcriticalȱ public,ȱ whichȱ wouldȱ notȱ beȱ theȱ caseȱ forȱ moreȱ sophisticatedȱ nonȬlinearȱ transformations,ȱ e.g.,ȱ normalization.ȱ Theȱ finalȱ scoreȱ usedȱ toȱ awardȱ freeȱ andȱ paidȱ placesȱ onȱ universityȱ coursesȱ isȱ calculatedȱ byȱ theȱ simpleȱ additionȱ ofȱ anȱ applicant’sȱ scoresȱ inȱ theȱ combinationȱ ofȱ subjectsȱ requiredȱ byȱ theȱ specificȱ universityȱ andȱspecialty.ȱ(SeeȱBoxȱ2ȱforȱanȱoverviewȱofȱtheȱUEȱsystem.)ȱ ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 17ȱ StudentsȱtakingȱtheȱUEȱdoȱnotȱhaveȱtoȱtakeȱtheȱGradeȱ12ȱstateȱgraduationȱexaminationȱconductedȱinȱ schools,ȱsinceȱpartȱAȱofȱtheȱUEȱprovidesȱthemȱwithȱaȱscoreȱforȱtheirȱgraduationȱdiploma.ȱ 18ȱ Theseȱcostsȱareȱbasedȱonȱtheȱ2014ȱATCȱregulationsȱforȱtheȱUE.ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 19ȱ Boxȱ2:ȱOverviewȱofȱtheȱUnifiedȱExaminationȱ Unified Examination and Supplementary Examinations UE and other Subjects examined: Armenian Language and Literature, Armenian History, World entrance History, Mathematics, Physics, Chemistry, Biology, Geography, Foreign Language examination (Russian, English, French, German, Spanish, Italian, or Persian). subjects Test format: Items capable of automated scoring through optical character recognition, i.e., multiple-choice and short (numerical)-answer questions (in Mathematics, Physics, and Geography). Duration: Foreign Language examinations = 150 minutes; all others = 180 minutes. Additional, centralized examinations are required in cases where speaking or writing skills have to be examined (e.g., for Journalism, Philology, Linguists, Foreign Languages, Literature, etc.). These are either ‘essays’ with a duration of 240 minutes, or oral tests (30 minutes per candidate). For certain specialties, some universities require a specialized ‘Internal Examination’ in which candidates have to perform or demonstrate skills, e.g., Fine Arts, Music and the Performing Arts, Sports, Design, etc. Organization ATC is responsible for all aspects of the UE, including the preparation of test papers, organizing the conduct of the examination in approved centers, and the marking/grading process. The main session for each subject examination is conducted on one day in 34 examination centers distributed all over the country. An additional session is organized for applicants who were registered, but absent on the day of the main examination for justified reasons. The organization and conduct of the examinations requires enormous resources. In 2013, a total of 863 people were involved in logistics, organization, and supervision of the UE. The biggest examinations are Armenian Language and Literature (8,647 candidates in 2013) and Mathematics (6,621). The smallest are in minority Foreign Languages, e.g., Spanish (20), Italian (4), and Persian (1 only!) Construction All UE tests contain 80 items (50 for Part A and 30 for Part B). Each item is scored of UE test 1 for correct answers or 0 for wrong answers, giving a maximum possible score of 80. and score An exception to this is the Armenian Language and Literature and Mathematics tests, processing which have some items that include six sequential sub-questions, each with three possible answers: ‘yes’, ‘no’, or ‘don’t know’. Each sub-question is scored 1 if correct, í1 if incorrect, or 0 if the chosen response is ‘don’t know’. The total raw score (maximum 80) is transformed to the 1–20 scale simply by dividing the total by four. The minimum passing grade is set at 7.5. In 2013, the overall failure rate was 28.4 percent of all test-takers. Examination Students registering for the UE are required to pay a fee of AMD 1500 for each fees subject (ATC 2014). Notes The new system was phased in over several years. In 2007, only Armenian Language and Literature was examined. In 2008, Mathematics and four foreign languages were added. In 2009, the suite of subjects was completed. The number of candidates has decreased significantly since 2007. For example, the number for Armenian Language and Literature has fallen from 14,682 to 8,934 (-39 percent) and for Mathematics, from 10,112 to 6,928 (-31 percent). The main reason for this is the decreasing school population. ȱ ȱ 20ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ CurrentȱPolicies,ȱStructures,ȱandȱResourcesȱforȱExaminationsȱ Theȱ Lawȱ onȱ Generalȱ Educationȱ (GORA,ȱ 2009)ȱ setsȱ outȱ theȱ generalȱ policyȱ concerningȱ endȬofȬphaseȱ schoolȱ graduationȱ examinations.ȱ Inȱ addition,ȱ theȱLawȱonȱHigherȱEducationȱ(GORA,ȱ2004)ȱestablishesȱtheȱprincipleȱthatȱ admissionȱ proceduresȱ toȱ HEIȱ atȱ theȱ Bachelor’sȱ degreeȱ levelȱ “shallȱ beȱ definedȱ byȱ theȱ Governmentȱ ofȱ theȱ Republicȱ ofȱ Armenia”19,ȱ i.e.,ȱ notȱ byȱ individualȱ universities.ȱ Inȱ bothȱ cases,ȱ implementationȱ ofȱ policyȱ isȱ delegatedȱ toȱ theȱ publicȱ bodiesȱ responsible—theȱ MOESȱ inȱ theȱ caseȱ ofȱ theȱ stateȱgraduationȱexaminations,ȱandȱtheȱATCȱforȱtheȱUE.ȱInȱtheȱcaseȱofȱUE,ȱ policiesȱ continueȱ toȱ evolve.ȱ First,ȱ inȱ 2009,ȱ ATCȱ wasȱ transferredȱ toȱ theȱ directȱ subordinationȱ ofȱ theȱ Governmentȱ Cabinet/Primeȱ Minister,ȱ therebyȱ creatingȱaȱdegreeȱofȱseparationȱfromȱtheȱMOES.ȱMoreȱrecently,ȱtheȱ‘Programȱ ofȱ Actionsȱ toȱ Fightȱ Corruptionȱ inȱ theȱ Educationȱ Sectorȱ 2011Ȭ2012’ȱ proposedȱ thatȱquestionȱpapersȱforȱtheȱUEȱshouldȱbeȱproducedȱbyȱtheȱselectionȱofȱitemsȱ fromȱ aȱ publiclyȬavailable,ȱ openȱ itemȱ bank.ȱ Thisȱ policy,ȱ implementedȱ byȱ ATCȱ inȱ 2012,ȱ isȱ effectivelyȱ aȱ returnȱ toȱ preȬ2007ȱ practices.ȱ Oneȱ consequenceȱofȱthisȱisȱthatȱallȱUEȱassessmentȱtasksȱareȱnow,ȱbyȱdefinition,ȱ setȱinȱfamiliarȱcontexts,ȱandȱhenceȱallȱproblemȱsolvingȱisȱroutine.ȱInȱshort,ȱ usingȱ anȱ openȱ itemȱ bankȱ reducesȱ theȱ opportunityȱ toȱ assessȱ higherȬlevelȱ cognitiveȱskillsȱandȱpromotesȱmemorizationȱandȱcoachingȱforȱtheȱtest.20ȱ ȱ Stateȱ graduationȱ examinationsȱ areȱ deliveredȱ byȱ theȱ MOESȱ (withȱ someȱ servicesȱ providedȱ byȱ ATC)ȱ throughȱ schoolȱ Directorsȱ andȱ adȱ hocȱ schoolȱ examinationȱ commissions.ȱ Few,ȱ ifȱ any,ȱ resourcesȱ areȱ explicitlyȱ dedicatedȱ toȱ theȱ preparationȱ andȱ conductȱ ofȱ theȱ examinations.ȱ Inȱ particular,ȱ theȱ administrationȱ ofȱ theȱ examinationsȱ andȱ theȱ scoringȱ ofȱ studentȱ responsesȱ areȱ consideredȱ partȱ ofȱ aȱ teacher’sȱ normalȱ workingȱ contractȱandȱsoȱnoȱextraȱpaymentȱisȱmade.ȱByȱwayȱofȱcontrast,ȱsignificantȱ resourcesȱ areȱ allocatedȱ toȱ carryingȱ outȱ theȱ UE.ȱ Fromȱ theȱ outsetȱ (2005),ȱ ATC’sȱ recurrentȱ expensesȱ haveȱ beenȱ fundedȱ directlyȱ throughȱ theȱ stateȱ budget.ȱ Initially,ȱ EQRPȱ Iȱ fundsȱ providedȱ theȱ capitalȱ expenditureȱ necessaryȱtoȱrefurbishȱsuitableȱaccommodation,ȱandȱtoȱprovideȱATCȱwithȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 19ȱ Articleȱ14,ȱparagraphȱ5.ȱ 20ȱ TheȱextensiveȱuseȱofȱprivateȱtutorsȱisȱconsideredȱaȱproblemȱinȱArmenia,ȱasȱelsewhere,ȱandȱ itȱ wasȱ hopedȱ thatȱ theȱ publicationȱ ofȱ itemȱ banksȱ wouldȱ reduceȱ theȱ pressureȱ onȱ parentsȱ toȱ employȱtutors.ȱUnfortunately,ȱanecdotalȱevidenceȱsuggestsȱthatȱcoachingȱisȱstillȱwidespreadȱ andȱ thatȱ tutorsȱ nowȱ trainȱ studentsȱ simplyȱ toȱ rememberȱ solutionsȱ ratherȱ thanȱ improvingȱ theirȱsubjectȬrelatedȱknowledgeȱandȱskillsȱ(Bagdasarayan,ȱpersonalȱcommunication,ȱ2014).ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 21ȱ technicalȱ equipment,ȱ includingȱ anȱ inȬhouseȱ printingȱ facilityȱ andȱ anȱ ITȱ systemȱ capableȱ ofȱ processingȱ theȱ new,ȱ technologicallyȬbasedȱ examinations.ȱ Whenȱ additionalȱ equipmentȱ needsȱ wereȱ identified,ȱ theȱ equipmentȱwasȱprovidedȱtoȱATCȱfromȱtheȱstateȱbudget.ȱThroughȱEQRPȱI,ȱ andȱ withȱ theȱ continuous,ȱ activeȱ supportȱ ofȱ GORA,ȱ Armeniaȱ wasȱ providedȱ withȱ aȱ “wellȱ functioningȱ institution,ȱ fullyȱ refurbishedȱ andȱ equippedȱwithȱmodernȱequipment…ȱ(with)ȱnineȱadministrativeȱdivisionsȱ andȱ 70ȱ staffȱ members”ȱ (CFEPȱ 2009,ȱ p.19).ȱ Anȱ overviewȱ ofȱ theȱ ATC’sȱ currentȱstatus,ȱfunctions,ȱandȱresourcesȱareȱprovidedȱinȱBoxȱ3.ȱ ȱ Boxȱ3:ȱOverviewȱofȱtheȱstatus,ȱfunctions,ȱandȱresourcesȱofȱtheȱATCȱ Status Legal entity, established by charter, with the status of a state, non-profit organization. ATC is directly subordinated to the Government Cabinet and hence the Prime Minister. Functions The operational objectives of ATC are defined by its charter. These include the development and conduct of all forms of student assessment in the general phase of education. In addition, the ATC is charged with providing methodological guidance and training to teachers “in the efficient use of new forms of assessment” (GORA 2004, Appendix, cl. 12[e]). ATC is also required to conduct analyses of the outcomes of various assessments and to provide GORA with reports. Under its charter, ATC is required to “maintain close links with schools and HEI and to provide results of examinations (at the student level)” (ibid. Appendix, cl. 12[j]). Specific responsibilities currently include: x Conduct of the UE at the secondary/tertiary interface x Preparation of test papers and scoring criteria for the state graduation examinations for Grades 9 and 12 x Preparation of model test papers to guide schools in the preparation of the state graduation examinations for Grade 4 x Preparation and conduct of annual, sample-based national achievement surveys x Preparation and conduct of annual, sample-based ‘external evaluations’ of student achievement (Grades 5-11) x Conduct of ILSA (currently TIMSS) Staffing ATC has a total staff of 81 (ATC data 2014). The staffing structure is shown below. Senior Management (Director, Deputies, and Advisor) 4 Finance and Accounting 4 Legal and Staff Administration Department 5 Examination Organization Division 12 ‘Testology’ Division 15 External Relations 9 ICT and Analysis Department 12 Publishing 8 Logistics and auxiliary staff 12 Financing The 2014 state budget line for ATC allocated AMD 413 million (~USD 1 million) for annual running costs, including salaries and utilities. Within this, AMD 167 million (~USD 400,000) were for ‘contractual works’, i.e., the development, production, and conduct of examinations and assessments. ATC also generates income through student fees for taking the UE. For 2013, this is estimated at AMD ~36 million (~USD 90,000). ȱ 22ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ MechanismsȱofȱChangeȱ Theȱ primaryȱ mechanismȱ forȱ buildingȱ institutionalȱ capacityȱ forȱ theȱ conductȱ ofȱ formalȱ examinationsȱ inȱ theȱ generalȱ phaseȱ ofȱ educationȱ wasȱ EQRPȱ I.ȱ Coordinatingȱ theȱ wideȱ rangeȱ ofȱ projectȱ inputsȱ associatedȱ withȱ establishingȱ theȱ ATC,ȱ andȱ simultaneouslyȱ developingȱ theȱ UEȱ modelȱ andȱ instruments,ȱ fellȱ toȱ theȱ Ministry’sȱ CFEP.ȱ Keyȱ elementsȱ includedȱ civilȱ worksȱ (refurbishingȱ aȱ buildingȱ toȱ provideȱ accommodationȱ forȱ ATC)ȱ andȱ theȱ procurementȱ ofȱ generalȱ andȱ specialistȱ equipment.ȱ Perhapsȱ theȱ mostȱ significantȱ taskȱ wasȱ coordinatingȱ andȱ checkingȱ theȱ qualityȱ ofȱ servicesȱ providedȱ byȱ internationalȱ consultantsȱ responsibleȱ forȱ developingȱ humanȱ resourcesȱthroughȱtheȱtrainingȱofȱsubjectȱspecialistsȱasȱitemȱwritersȱandȱATCȱ professionalȱstaff.ȱAȱsummaryȱofȱtheseȱactivitiesȱandȱrelatedȱoutcomesȱcanȱbeȱ foundȱinȱtheȱcompletionȱreportȱforȱtheȱprojectȱ(CFEP,ȱ2009).ȱ Initially,ȱadvocatesȱofȱtheȱreformsȱmetȱwithȱcautiousȱresistanceȱfromȱ voicesȱ withinȱ theȱ Government.ȱ Inȱ particular,ȱ theȱ Ministryȱ ofȱ Financeȱ expressedȱ graveȱ concernsȱ overȱ proposalsȱ toȱ establishȱ aȱ new,ȱ specializedȱ institutionȱ atȱ aȱ timeȱ ofȱ generalȱ budgetaryȱ restraint,ȱ especiallyȱ whenȱ studentȱ assessmentȱ hadȱ traditionallyȱ beenȱ conductedȱ withoutȱ anyȱ explicitȱ supportȱ fromȱ theȱ stateȱ budget.ȱ Inȱ orderȱ toȱ overcomeȱ theseȱ understandableȱ objections,ȱ theȱ MOESȱ andȱ theȱ CFEPȱ hadȱ toȱ prepareȱ wellȬfoundedȱ justificationsȱ setȱ outȱ inȱ aȱ rangeȱ ofȱ formalȱ documents.ȱ Inȱ addition,ȱ aȱ seriesȱ ofȱ meetingsȱ wasȱ heldȱ withȱ relevantȱ Ministriesȱ andȱ theȱ CabinetȱofȱtheȱGORAȱinȱorderȱtoȱexplainȱtheȱrationaleȱbehindȱtheȱreformsȱ andȱ theȱ potentialȱ longȬtermȱ benefits.ȱ Withoutȱ thisȱ advocacy,ȱ itȱ isȱ highlyȱ unlikelyȱthatȱATCȱwouldȱhaveȱcomeȱintoȱexistence.ȱ ȱ Onceȱ theȱ ATCȱ wasȱ formallyȱ established,ȱ identifyingȱ andȱ recruitingȱ theȱ rightȱ personnelȱ emergedȱ asȱ aȱ majorȱ challenge.ȱ Aȱ Directorȱ wasȱ recruitedȱwithȱtheȱnecessaryȱmanagementȱskills,ȱbutȱthereȱwasȱaȱshortageȱ ofȱexperiencedȱassessmentȱandȱIT/analyticalȱspecialists.ȱToȱovercomeȱthis,ȱ peopleȱ withȱ theȱ potentialȱ toȱ becomeȱ specialistsȱ wereȱ appointedȱ andȱ thenȱ supportedȱ underȱ EQRPȱ Iȱ throughȱ trainingȱ andȱ ‘learningȱ onȱ theȱ job’.ȱ Inȱ particular,ȱ leadingȱ testȱ developersȱ emergedȱ fromȱ theȱ project’sȱ subjectȱ workingȱgroups,ȱwhichȱhadȱbeenȱformedȱfromȱtheȱbestȱsubjectȱspecialistsȱ fromȱ respectedȱ schools,ȱ universities,ȱ andȱ theȱ NIE.ȱ However,ȱ itȱ shouldȱ beȱ notedȱ thatȱ recruitingȱ andȱ retainingȱ theȱ bestȱ technicalȱ staffȱ inȱ theȱ ATCȱ remainsȱaȱchallenge.ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 23ȱ Sinceȱ theȱ ATCȱ hasȱ beenȱ fullyȱ operational,ȱ itȱ hasȱ becomeȱ aȱ keyȱ mechanismȱforȱenhancingȱtheȱscopeȱandȱqualityȱofȱtheȱoverallȱassessmentȱ systemȱ inȱ Armenia.ȱ Thisȱ isȱ largelyȱ dueȱ toȱ twoȱ keyȱ factors.ȱ First,ȱ asȱ theȱ organizationȱ hasȱ provedȱ itsȱ competence,ȱ theȱ MOESȱ hasȱ foundȱ itȱ increasinglyȱ convenientȱ toȱ transferȱ responsibilityȱ forȱ examinationsȱ andȱ otherȱ formsȱ ofȱ assessmentȱ toȱ ATC.ȱ Second,ȱ buildingȱ onȱ capacitiesȱ developedȱ throughȱ consultancyȱ andȱ trainingȱ servicesȱ suppliedȱ underȱ EQRPȱ I,ȱ theȱ professionalȱ staffȱ ofȱ ATCȱ areȱ nowȱ initiatingȱ qualitativeȱ reforms.ȱ Inȱ particular,ȱ thereȱ areȱ signsȱ thatȱ ATCȱ isȱ strengtheningȱ itsȱ analysis,ȱreporting,ȱandȱfeedbackȱmechanisms.ȱ DriversȱofȱChangeȱ Inȱ Armenia,ȱ asȱ inȱ manyȱ otherȱ formerȱ socialistȱ republics,ȱ reformsȱ inȱ theȱ fieldȱ ofȱ examinationsȱ wereȱ givenȱ impetusȱ byȱ twoȱ powerfulȱ internalȱ drivers.ȱ First,ȱ GORAȱ andȱ theȱ MOESȱ recognizedȱ theȱ importanceȱ ofȱ reformingȱ highȬstakeȱ examinationsȱ inȱ orderȱ toȱ reinforceȱ broaderȱ effortsȱ beingȱ madeȱ toȱ improveȱ theȱ contentȱ andȱ deliveryȱ ofȱ theȱ nationalȱ curriculumȱ toȱ makeȱ itȱ moreȱ relevantȱ toȱ aȱ modern,ȱ global,ȱ competitiveȱ environment.ȱ Second,ȱ thereȱ wasȱ greatȱ concernȱ thatȱ traditionalȱ universityȱ admissionsȱ proceduresȱ wereȱ notȱ onlyȱ inequitableȱ andȱ opaque,ȱ butȱ alsoȱ proneȱ toȱ corruptionȱ andȱ malpractice.ȱ Inȱ short,ȱ thereȱ wasȱ aȱ clearȱ understandingȱ ofȱ theȱ roleȱ thatȱ suchȱ examinationsȱ playȱ bothȱ withinȱ theȱ educationalȱenvironmentȱandȱinȱsocietyȱasȱaȱwhole.ȱThisȱwasȱinȱtuneȱwithȱ theȱ viewȱ ofȱ internationalȱ agenciesȱ thatȱ externalȱ examinationsȱ provideȱ aȱ powerfulȱ leverȱ forȱ bringingȱ aboutȱ desirableȱ changesȱ inȱ anȱ educationȱ system.ȱThisȱmayȱbeȱsummarizedȱas:ȱ“Goodȱexaminationȱdesignȱcanȱleadȱ toȱsocialȱcohesion,ȱitȱcanȱhelpȱchooseȱtalentȱfairly,ȱitȱcanȱengenderȱtrustȱinȱ publicȱ institutions,ȱ andȱ itȱ canȱ giveȱ confidenceȱ inȱ theȱ generalȱ conductȱ ofȱ theȱ publicȱ schoolȱ system”ȱ (Heynemanȱ 2009,ȱ p.11).ȱ Theȱ alignmentȱ ofȱ internalȱandȱexternalȱdriversȱensuredȱthatȱtheȱdesignȱofȱEQRPȱIȱallocatedȱ sufficientȱ human,ȱ physical,ȱ andȱ financialȱ resourcesȱ forȱ buildingȱ aȱ newȱ examinationȱ systemȱ and,ȱ ofȱ supremeȱ importance,ȱ aȱ newȱ organizationȱ toȱ deliverȱthatȱsystem.ȱ Inevitably,ȱ aȱ tensionȱ existsȱ betweenȱ designingȱ examinationsȱ toȱ enhanceȱ educationalȱ qualityȱ andȱ designingȱ themȱ toȱ fightȱ corruption.ȱ Inȱ Armenia,ȱtheȱmostȱimportantȱdriverȱinȱtheȱintroductionȱofȱtheȱUEȱwasȱtheȱ needȱtoȱmaintainȱsecurityȱandȱeliminate,ȱasȱfarȱasȱpossible,ȱmalpracticeȱinȱ ȱ 24ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ theȱ assessmentȱ ofȱ studentsȱ andȱ hence,ȱ theȱ universityȱ admissionsȱ system.ȱ Therefore,ȱ inȱ commonȱ withȱ manyȱ otherȱ transitionalȱ countriesȱ inȱ theȱ region,ȱ allȱ UEȱ testsȱ consistȱ entirelyȱ ofȱ objective,ȱ selectionȬtypeȱ itemsȱ wherebyȱ students’ȱ selectedȱ responsesȱ canȱ beȱ recognizedȱ byȱ opticalȱ characterȱ recognitionȱ softwareȱ andȱ scoredȱ byȱ computerȱ systems.ȱ Theȱ limitationsȱ ofȱ thisȱ approach,ȱ andȱ theȱ likelyȱ negativeȱ impactȱ onȱ teaching,ȱ wereȱ wellȱ understoodȱ byȱ thoseȱ whoȱ designedȱ theȱ system.ȱ However,ȱ inȱ theȱ earlyȱ years,ȱ itȱ wasȱ essentialȱ thatȱ theȱintegrityȱofȱ theȱ systemȱshouldȱ beȱ maintainedȱandȱsoȱtheȱexaminationsȱwereȱintroducedȱinȱthisȱ‘safeȱmode’.ȱ Nowȱ thatȱ theȱ systemȱ hasȱ becomeȱ embeddedȱ andȱ gainedȱ aȱ degreeȱ ofȱ publicȱ trust,ȱ someȱ commentatorsȱ haveȱ suggestedȱ thatȱ changesȱ shouldȱ beȱ introducedȱ toȱ increaseȱ theȱ educationalȱ validityȱ ofȱ theȱ examination,ȱ butȱ theȱ tensionȱ betweenȱ maintainingȱ securityȱ andȱ enhancingȱ educationalȱ valueȱisȱstillȱstrong.ȱThisȱisȱcapturedȱinȱtheȱfollowingȱobservation:ȱ ȱ Thereȱ isȱ …ȱ aȱ seriousȱ riskȱ thatȱ theȱ educationalȱ valueȱ ofȱ theȱ examȱ willȱ notȱ beȱ consideredȱ theȱ mainȱ priority,ȱ andȱ questionȱ typesȱ whichȱ canȱ promoteȱ studentȱ learningȱ mayȱ beȱ neglected.ȱ Theseȱ includeȱ questionsȱ orȱ tasksȱ requiringȱ oralȱ fluency,ȱ skillsȱ inȱ solvingȱ nonȬroutineȱ problemsȱ andȱ problemsȱ relatedȱ toȱ “realȬworld”ȱ situations,ȱ writingȱ andȱ practicalȱ work.ȱManyȱofȱtheseȱlatterȱskillsȱareȱregardedȱasȱessentialȱforȱpreparingȱ studentsȱ forȱ theȱ knowledgeȱ economyȱ andȱ forȱ theȱ developmentȱ ofȱ importantȱ “softȱ skills.”ȱ Theȱ priorityȱ givenȱ toȱ securityȱ couldȱ leadȱ toȱ anȱ examinationȱ systemȱ dominatedȱ byȱ theȱ exclusiveȱ useȱ ofȱ computerȬscoredȱmultipleȬchoiceȱquestionsȱ(READȱ2012,ȱp.ȱ7Ȭ8).ȱ Unfortunately,ȱ fromȱ anȱ educationalȱ viewpoint,ȱ itȱ appearsȱ thatȱ theȱ obsessionȱ withȱ maintainingȱ securityȱ andȱ eliminatingȱ allȱ subjectivityȱ mayȱ beȱ winningȱ theȱ argument.ȱ Forȱ example,ȱ asȱ describedȱ previously,ȱ allȱ objectiveȱitemsȱincludedȱinȱUEȱquestionȱpapersȱareȱnowȱselectedȱfromȱanȱ openȱ itemȱ bank,ȱ effectivelyȱ eliminatingȱ theȱ opportunityȱ toȱ testȱ higherȬlevelȱskillsȱinȱnovel,ȱauthenticȱcontexts.ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 25ȱ LargeȬScaleȱAssessmentsȱ InternationalȱLargeȬScaleȱAssessmentsȱ Armeniaȱ participatedȱ inȱ theȱ 2003,ȱ 2007,ȱ andȱ 2011ȱ cyclesȱ ofȱ TIMSS,ȱ withȱ samplesȱ beingȱ drawnȱ forȱ bothȱ theȱ Gradeȱ 4ȱ andȱ Gradeȱ 8ȱ populations.21ȱ Itȱ isȱ alsoȱ participatingȱ inȱ theȱ 2015ȱ cycle.ȱ Theȱ mainȱ purposesȱ ofȱ Armenia’sȱ participationȱ inȱ TIMSSȱ are:ȱ (a)ȱ toȱ giveȱ aȱ snapshotȱ ofȱ whatȱ Armenianȱ studentsȱ knowȱ andȱ canȱ doȱ inȱ theȱ subjectȱ areasȱ ofȱ Mathematicsȱ andȱ ScienceȱatȱGradesȱ4ȱandȱ8;ȱ(b)ȱtoȱprovideȱdataȱthatȱcanȱbeȱusedȱtoȱidentifyȱ trendsȱinȱeducationalȱperformance,ȱi.e.,ȱtoȱtrackȱprogressȱoverȱtime;ȱ(c)ȱtoȱ allowȱ theȱ achievementȱ levelsȱ ofȱ Armenianȱ studentsȱ toȱ beȱ comparedȱ withȱ thoseȱofȱtheirȱpeersȱinȱotherȱcountries;ȱandȱ(d)ȱtoȱidentifyȱfactorsȱthatȱmayȱ promoteȱ orȱ hinderȱ educationalȱ progressȱ soȱ thatȱ theseȱ canȱ beȱ accountedȱ forȱinȱnationalȱpolicies,ȱstrategies,ȱandȱpractices.ȱ Inȱ TIMSS,ȱ asȱ inȱ allȱ majorȱ international,ȱ largeȬscaleȱ assessmentsȱ (ILSA),ȱtheȱhighlyȱtechnicalȱprocessȱofȱdrawingȱaȱrepresentativeȱsampleȱisȱ carriedȱ outȱ byȱ anȱ internationalȱ contractor,ȱ usingȱ theȱ samplingȱ frameȱ providedȱ byȱ theȱ countryȱ inȱ question.ȱ Aȱ typicalȱ sampleȱ forȱ Armeniaȱ atȱ eachȱ populationȱ assessedȱ isȱ 150ȱ schoolsȱ andȱ approximatelyȱ 5,500ȱ students.ȱ Forȱ example,ȱ Armenia’sȱ achievedȱ sampleȱ inȱ TIMSSȱ 2011ȱ wasȱ 5,146ȱ studentsȱ inȱ 150ȱ schoolsȱ forȱ Gradeȱ 4,ȱ andȱ 5,846ȱ studentsȱ inȱ 153ȱ schoolsȱforȱGradeȱ8ȱ(IEA,ȱ2012).ȱ ȱ Theȱ testsȱ areȱ developedȱ byȱ theȱ internationalȱ contractorȱ accordingȱ toȱ theȱ TIMSSȱ assessmentȱ frameworksȱ andȱ procedures.22ȱ Itemsȱ areȱ translatedȱ andȱ locallyȱ fieldȱ tested.ȱ Theȱ degreeȱ toȱ whichȱ theȱ TIMSSȱ assessmentȱ frameworksȱareȱalignedȱwithȱnationalȱcurriculaȱvariesȱacrossȱcountries.ȱInȱ Armenia,ȱ theȱ correspondenceȱ inȱ Mathematicsȱ isȱ relativelyȱ good,ȱ especiallyȱ atȱ theȱ Gradeȱ 8ȱ levelȱ whereȱ allȱ 19ȱ topicsȱ inȱ theȱ 2011ȱ TIMSSȱ frameworkȱ wereȱ explicitlyȱ requiredȱ byȱ theȱ nationalȱ curriculum.ȱ AlignmentȱisȱlessȱcloseȱforȱScience,ȱbutȱisȱstillȱconsideredȱadequate.ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 21ȱ Armeniaȱ alsoȱ tookȱ partȱ inȱ theȱ TIMSSȱ Advanced21ȱ studyȱ ofȱ 2008,ȱ whichȱ isȱ targetedȱ atȱ studentsȱ studyingȱ Physicsȱ andȱ Mathematicsȱ inȱ theȱ finalȱ yearȱ ofȱ compulsoryȱ generalȱ education.ȱ 22ȱ Assessmentȱframeworksȱareȱavailableȱat:ȱhttp://timssandpirls.bc.edu/timss2015ȱ /frameworks.htmlȱ ȱ 26ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ Theȱ dataȱ forȱ Armeniaȱ presentedȱ inȱ theȱ internationalȱ reportsȱ ofȱ theȱ InternationalȱAssociationȱ forȱ theȱ Evaluationȱ ofȱ Educationalȱ Achievementȱ (IEA)ȱ areȱ theȱ mainȱ sourceȱ ofȱ informationȱ usedȱ toȱ reportȱ theȱ nation’sȱ performanceȱ toȱ policyȱ makers,ȱ educationalȱ practitioners,ȱ andȱ theȱ widerȱ public. 23 ȱ Forȱ example,ȱ dataȱ extractedȱ fromȱ theȱ internationalȱ reportȱ isȱ usedȱ toȱ disseminateȱ headlineȱ resultsȱ viaȱ theȱ mediaȱ andȱ throughȱ conferencesȱandȱseminarsȱaimedȱatȱeducationalȱprofessionals.ȱ Aȱ nationalȱ reportȱ onȱ Armenia’sȱ performanceȱ inȱ TIMSSȱ isȱ producedȱ soonȱ afterȱ completionȱ ofȱ theȱ testȱ administrationȱ andȱ scoringȱ phase,ȱ andȱ beforeȱ theȱ internationalȱ reportȱ isȱ released.24ȱ Theȱ reportȱ includesȱ nationalȱ distributionsȱofȱstudentȱachievementȱinȱMathematicsȱandȱSciencesȱasȱrawȱ scores,ȱ andȱ aȱ comparisonȱ ofȱ theȱ resultsȱ byȱ administrativeȱ regionȱ (marz);ȱ correlationsȱ betweenȱ achievementȱ andȱ backgroundȱ factors;ȱ andȱ comparisonȱ withȱ previousȱ TIMSSȱ resultsȱ priorȱ toȱ scaling,ȱ i.e.,ȱ rawȱ scores. 25 ȱ Sinceȱ allȱ preliminaryȱ analysesȱ areȱ basedȱ onȱ rawȱ scores,ȱ noȱ comparisonȱ ofȱ Armenia’sȱ resultsȱ withȱ thoseȱ ofȱ otherȱ countriesȱ isȱ included.ȱ Thisȱ reportȱisȱ notȱ widelyȱ circulatedȱ andȱ itȱisȱ unclearȱ whetherȱ itȱ hasȱmuchȱofȱanȱimpactȱonȱpolicyȱmakersȱorȱpractitioners.ȱ NationalȱLargeȬScaleȱAssessmentsȱ Buildingȱ onȱ experienceȱ gatheredȱ throughȱ TIMSS,ȱ Armeniaȱ isȱ nowȱ implementingȱaȱNLSAȱprogramȱacrossȱthreeȱdomains:ȱArmenianȱStudiesȱ (Armenianȱ Languageȱ andȱ Literature,ȱ andȱ Armenianȱ History),ȱ Scienceȱ (Physics/Chemistryȱ andȱ Geography/Biology),ȱ andȱ Foreignȱ Languagesȱ (Englishȱ andȱ Russian).ȱ Fourȱ surveysȱ haveȱ beenȱ completedȱ toȱ date:ȱ Armenianȱ Studiesȱ (HAAS)ȱ inȱ 2010,ȱ Physicsȱ andȱ Chemistryȱ (BAAS)ȱ inȱ 2011,ȱ Geographyȱ andȱ Biologyȱ (BAAS)ȱ inȱ 2012,ȱ andȱ Foreignȱ Languagesȱ (OLAAS)ȱ inȱ 2013. 26 ȱ NLSAȱ studiesȱ differȱ considerablyȱ fromȱ allȱ otherȱ assessmentsȱ previouslyȱ conductedȱ inȱ Armeniaȱ inȱ theirȱ purposes,ȱ design,ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 23ȱ TIMSSȱinternationalȱreportsȱareȱavailableȱat:ȱhttp://timssandpirls.bc.edu/iscȱ /publications.htmlȱ ȱ 24ȱ TheȱnationalȱreportȱisȱavailableȱonlyȱinȱArmenian.ȱ 25 ȱ Rawȱ scoresȱ areȱ simpleȱ countsȱ ofȱ creditsȱ achievedȱ forȱ correctȱ andȱ partiallyȬcorrectȱ responses.ȱ Forȱ internationalȱ reporting,ȱ theseȱ areȱ scaledȱ throughȱ theȱ applicationȱ ofȱ complexȱ itemȱ responseȱ modelsȱ andȱ weightedȱ toȱ ensureȱ thatȱ averagesȱ areȱ trulyȱ representativeȱ ofȱ theȱ population.ȱ ȱ 26ȱ HAAS,ȱ BAAS,ȱ andȱ OLAASȱ areȱ theȱ localȱ acronymsȱ usedȱ respectivelyȱ forȱ theȱ NLSAȱ inȱ ArmenianȱStudies,ȱSciences,ȱandȱForeignȱLanguages.ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 27ȱ andȱ instruments.ȱ Inȱ particular,ȱ theȱ itemsȱ areȱ notȱ limitedȱ toȱ assessingȱ knowledgeȱ fromȱ theȱ curriculum,ȱ butȱ alsoȱ assessȱ theȱ practicalȱ applicationȱ ofȱ knowledgeȱ andȱ skillsȱ relevantȱ toȱ everydayȱ life.ȱ Theȱ HAASȱ andȱ BAASȱ testsȱ includeȱ multipleȬchoice,ȱ shortȬanswer,ȱ andȱ openȬendedȱ questionsȱ whilstȱtheȱOLAASȱtestsȱincludeȱtextȬbasedȱitemsȱsimilarȱtoȱthoseȱusedȱin,ȱ forȱexample,ȱPIRLS.ȱ Theȱ toolsȱ forȱ NLSAȱ followȱ theȱ generalȱ modelȱ usedȱ inȱ TIMSSȱ inȱ thatȱ eachȱ studentȱ selectedȱ toȱ takeȱ theȱ testȱ completesȱ threeȱ parts:ȱ aȱ testȱ forȱ theȱ firstȱsubject,ȱaȱtestȱforȱtheȱsecondȱsubject,ȱandȱaȱquestionnaireȱdesignedȱtoȱ collectȱ informationȱ onȱ studentȱ background.ȱ Informationȱ aboutȱ NLSAȱ instrumentsȱisȱavailableȱonȱtheȱATCȱwebsite,ȱincludingȱsomeȱexamplesȱofȱ theȱtasksȱpresentedȱtoȱstudents.ȱ ȱ Typically,ȱ sixȱ testȱ bookletsȱ linkedȱ byȱ commonȱ (anchor)ȱ itemsȱ areȱ usedȱ forȱ eachȱ NLSA.ȱ Stratifiedȱ samplesȱ areȱ selected,ȱ butȱ notȱ byȱ probabilisticȱ methods.ȱ Analysisȱ ofȱ resultsȱ focusesȱ onȱ rawȱ scoresȱ (e.g.,ȱ percentageȱ correct),ȱ butȱ itemȱ responseȱ theoryȱ (IRT)ȱ isȱ usedȱ toȱ ascertainȱ theȱpsychometricȱpropertiesȱofȱitems.ȱTechnicalȱreportsȱareȱproduced,ȱbutȱ theseȱ areȱ predominantlyȱ forȱ ATC’sȱ internalȱ use.ȱ Theyȱ areȱ availableȱ toȱ MOESȱ staff,ȱ butȱ areȱ notȱ yetȱ usedȱ toȱ informȱ teachersȱ andȱ otherȱ practitioners.ȱ Itȱ shouldȱ beȱ notedȱ thatȱ ATCȱ considersȱ itsȱ workȱ thusȱ farȱ onȱ NLSAȱ toȱ beȱ “researchȱ andȱ development—justȱ theȱ firstȱ stepȱ towardsȱ anȱ effectiveȱNLSA”ȱ(Bagdasaryan,ȱpersonalȱcommunication,ȱ2014).ȱ Inȱ additionȱ toȱ theȱ nationalȱ assessmentsȱ describedȱ above,ȱ anotherȱ formȱ ofȱ sampleȬbasedȱ assessment—soȬcalledȱ ‘Externalȱ Evaluations’—hasȱ beenȱ carriedȱ outȱ byȱ ATCȱ sinceȱ 2010.ȱ Theȱ purposeȱ ofȱ theseȱ evaluationsȱ isȱ toȱ provideȱ objectiveȱ dataȱ aboutȱ studentȱ achievementȱ inȱ differentȱ subjectȱ areasȱ thatȱ canȱ beȱ comparedȱ withȱ theȱ resultsȱ fromȱ moreȱ subjective,ȱ internalȱ schoolȱ assessments.ȱ Theȱ evaluationsȱ coverȱ Gradesȱ 5Ȭ11,ȱ withȱ subjectsȱ andȱ gradesȱ toȱ beȱ testedȱ changingȱ annually.ȱ Theȱ testingȱ instrumentsȱ areȱ closelyȱ basedȱ onȱ theȱ curriculum,ȱ andȱ areȱ veryȱ similarȱ toȱ theȱ testsȱ usedȱ forȱ theȱ stateȱ graduationȱ examinationsȱ inȱ theirȱ structure,ȱ methods,ȱandȱscoringȱprocedures.ȱHowever,ȱtheyȱareȱappliedȱtoȱaȱsampleȱ ofȱstudentsȱfromȱaȱlimitedȱnumberȱofȱschools.ȱPragmatic,ȱnonȬprobabilityȱ samplingȱ methodsȱ areȱ used,ȱ butȱ theȱ sampleȱ isȱ consideredȱ toȱ beȱ adequatelyȱ representativeȱofȱ theȱ populationȱ inȱ termsȱofȱschoolȱ type,ȱsize,ȱ andȱlocation.ȱInȱadditionȱtoȱtests,ȱquestionnairesȱareȱcompletedȱbyȱschoolȱ ȱ 28ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ directorsȱ andȱ teachersȱ inȱ orderȱ toȱ investigateȱ theȱ relationshipȱ betweenȱ studentȱachievementȱandȱcertainȱbackgroundȱfactors.ȱ ȱ Inȱ 2013,ȱ theȱ Externalȱ Evaluationȱ wasȱ conductedȱ inȱ 63ȱ schoolsȱ selectedȱ proportionallyȱ fromȱ allȱ marzesȱ (administrativeȱ regions).ȱ Approximatelyȱ 2,700ȱ studentsȱ participatedȱ inȱ testsȱ ofȱ Armenianȱ Languageȱ andȱ Literature,ȱ andȱ Mathematics.ȱ 267ȱ teachersȱ wereȱ alsoȱ includedȱinȱtheȱsurvey.ȱEachȱsubjectȱtestȱlastedȱ60ȱminutesȱandȱcomprisedȱ 20ȱ dichotomouslyȬscoredȱ items.ȱ Theȱ scoringȱ scaleȱ wasȱ 0Ȭ20ȱ points,ȱ withȱ finalȱresultsȱtransformedȱtoȱtheȱubiquitousȱ10Ȭpointȱreportingȱscale.ȱ Theȱ resultsȱ ofȱ Externalȱ Evaluationsȱ areȱ analyzedȱ andȱ reportsȱ areȱ produced.ȱ Theȱ 2013ȱ reportȱ isȱ availableȱ onȱ theȱ ATCȱ website. 27 ȱ Theȱ findingsȱ ofȱ theȱ Externalȱ Evaluationȱ areȱ usuallyȱ discussedȱ atȱ theȱ Ministryȱ Collegium.ȱ Itȱ isȱ difficultȱ toȱ knowȱ what,ȱ ifȱ any,ȱ impactȱ theȱ informationȱ fromȱthisȱformȱofȱassessmentȱhasȱonȱtheȱwiderȱsystem.ȱ CurrentȱPolicies,ȱStructures,ȱandȱResourcesȱforȱILSAȱandȱNLSAȱ Governmentȱ policyȱ regardingȱ ILSAȱ indicatesȱ onlyȱ thatȱ Armeniaȱ willȱ participateȱ inȱ theȱ 2015ȱ cycleȱ ofȱ TIMSS.ȱ Inȱ recentȱ years,ȱ itȱ hasȱ beenȱ proposedȱ thatȱ Armeniaȱ shouldȱ alsoȱ takeȱ partȱ inȱ theȱ Progressȱ inȱ Internationalȱ Readingȱ Literacyȱ Studyȱ (PIRLS)ȱ 2016ȱ andȱ theȱ Organizationȱ forȱ Economicȱ Cooperationȱ andȱ Developmentȱ (OECD)ȱ Programȱ forȱ Internationalȱ Studentȱ Assessmentȱ (PISA).ȱ However,ȱ theseȱ haveȱ notȱ beenȱ approvedȱ byȱ theȱ GORAȱ andȱ soȱ theȱ deȱ factoȱ policyȱ isȱ that,ȱ forȱ theȱ foreseeableȱ future,ȱ Armenia’sȱ participationȱ inȱ ILSAȱ willȱ beȱ limitedȱ toȱTIMSS.ȱ ATCȱ isȱ theȱ institutionȱ responsibleȱ forȱ theȱ design,ȱ conduct,ȱ analysis,ȱ andȱreportingȱofȱTIMSSȱandȱallȱformsȱofȱnationalȱassessment.ȱItȱhasȱaȱstaffȱ ofȱ 81,ȱ includingȱ 15ȱ assessmentȱ specialists,ȱ whoȱ workȱ onȱ bothȱ NLSAȱ andȱ ILSAȱ activitiesȱ accordingȱ toȱ need.ȱ Itsȱ structureȱ isȱ organizedȱ byȱ staffȱ functionsȱ andȱ expertiseȱ ratherȱ thanȱ accordingȱ toȱ theȱ typeȱ ofȱ assessmentȱ andȱ soȱ itȱ isȱ difficultȱ toȱ estimateȱ withȱ precisionȱ theȱ humanȱ resourcesȱ associatedȱwithȱspecificȱlargeȬscaleȱassessments.28ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 27ȱ AvailableȱonlyȱinȱArmenian.ȱ 28ȱ Thisȱisȱnotȱtheȱcaseȱin,ȱforȱexample,ȱLithuaniaȱwhereȱaȱdedicatedȱunitȱforȱILSAȱsitsȱwithinȱ theȱNationalȱExaminationsȱCentre.ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 29ȱ ATCȱ hasȱ toȱ fundȱ allȱ NLSAȱ activitiesȱ fromȱ itsȱ totalȱ budgetaryȱ allocationȱ forȱ contractualȱ worksȱ (i.e.,ȱ productionȱ andȱ processingȱ ofȱ examinationsȱ andȱ largeȬscaleȱ assessments).ȱ Inȱ 2013,ȱ theȱ provisionȱ forȱ allȱ suchȱ worksȱ wasȱ AMDȱ 167ȱ millionȱ (~USDȱ 400,000).ȱ Itȱ isȱ difficultȱ toȱ estimateȱexactlyȱhowȱmuchȱofȱthatȱwasȱallocatedȱtoȱaȱparticularȱNLSA.ȱ Forȱ aȱ relativelyȱ poorȱ countryȱ suchȱ asȱ Armenia,ȱ participationȱ inȱ TIMSSȱ isȱ expensive.ȱ Inȱ additionȱ toȱ theȱ standardȱ internationalȱ feesȱ requiredȱ forȱ participationȱ inȱ theȱ exercise,ȱ thereȱ areȱ significantȱ nationalȱ costsȱ associatedȱ withȱ translation,ȱ printing,ȱ packingȱ andȱ distribution,ȱ scoring,ȱ andȱ dataȱ entry.ȱ Forȱ example,ȱ Armenia’sȱ nationalȱ budgetȱ forȱ theȱ 2011ȱ TIMSSȱ cycleȱ wasȱ AMDȱ 15,450,000,ȱ equivalentȱ toȱ aboutȱ USDȱ 40,500.ȱ ApproximatelyȱoneȬthirdȱofȱthisȱwasȱusedȱforȱfieldȱtestingȱandȱtwoȬthirdsȱ forȱ theȱ mainȱ dataȱ collection.ȱ Inȱ additionȱ toȱ theseȱ costs,ȱ thereȱ areȱ significantȱhiddenȱcostsȱassociatedȱwithȱservicesȱprovidedȱ‘freeȱofȱcharge’ȱ by,ȱ forȱ example,ȱ regionalȱ educationalȱ offices,ȱ schools,ȱ principals,ȱ andȱ teachers.ȱ MechanismsȱofȱChangeȱ Theȱ mainȱ vehicleȱ forȱ establishingȱ theȱ foundationsȱ ofȱ Armenia’sȱ comprehensiveȱstudentȱassessmentȱsystem—includingȱILSA—wasȱEQRPȱ I,ȱ whichȱ wasȱ largelyȱ fundedȱ fromȱ aȱ Worldȱ Bankȱ creditȱ agreedȱ inȱ 2003.ȱ EQRPȱ Iȱ providedȱ resourcesȱ forȱ buildingȱ operationalȱ capacityȱ andȱ establishingȱ theȱ ATC,ȱ whichȱ eventuallyȱ wouldȱ beȱ availableȱ toȱ carryȱ outȱ systemȱ monitoringȱ assessments.ȱ Itȱ alsoȱ providedȱ essentialȱ fundingȱ forȱ Armenia’sȱparticipationȱinȱtheȱ2003ȱcycleȱofȱTIMSS.ȱ Priorȱ toȱ ATCȱ becomingȱ aȱ fullyȬfunctioningȱ organization,ȱ theȱ CFEPȱ wasȱ theȱ mechanismȱ throughȱ whichȱ theȱ 2003ȱ andȱ 2007ȱ cyclesȱ ofȱ TIMSSȱ wereȱ conducted.ȱ Thisȱ wasȱ aȱ veryȱ unusualȱ arrangementȱ givenȱ thatȱ theȱ implementationȱagencyȱisȱusuallyȱaȱtechnicalȱinstitutionȱ(e.g.,ȱaȱuniversityȱ departmentȱ orȱ aȱ nationalȱ assessmentȱ agency).ȱ Inȱ anyȱ event,ȱ theȱ CFEPȱ collaboratedȱ effectivelyȱ withȱ theȱ Nationalȱ Researchȱ Coordinatorȱ (NRC)ȱ andȱ wasȱ successfulȱ inȱ commissioningȱ allȱ servicesȱ necessaryȱ toȱ prepareȱ materialsȱ andȱ conductȱ testingȱ activitiesȱ inȱ theȱ field.ȱ Thisȱ transitionalȱ arrangementȱ provedȱ highlyȱ effectiveȱ andȱ onceȱ ATCȱ wasȱ operational,ȱ onȬgoingȱresponsibilityȱforȱTIMSSȱwasȱtransferredȱtoȱit.ȱ ȱ Theȱ mechanismȱ forȱ fundingȱ ILSAȱ hasȱ changedȱ overȱ time.ȱ Initially,ȱ ArmeniaȱwasȱexemptedȱbyȱtheȱIEAȱfromȱtheȱpaymentȱofȱtheȱinternationalȱ ȱ 30ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ costsȱ associatedȱ withȱ participationȱ inȱ TIMSS,ȱ withȱ Worldȱ Bankȱ projectȱ fundsȱcoveringȱtheȱremainingȱnationalȱcostsȱforȱTIMSSȱ2003ȱandȱ2007.ȱForȱ theȱ firstȱ halfȱ ofȱ theȱ TIMSSȱ 2011ȱ cycle,ȱ internationalȱ costsȱ wereȱ paidȱ fromȱ EQRPȱIȱfunds.ȱSubsequently,ȱnationalȱcostsȱforȱtheȱfieldȱtestingȱandȱmainȱ dataȱcollectionȱactivitiesȱandȱinternationalȱcostsȱforȱtheȱsecondȱhalfȱofȱtheȱ cycleȱ wereȱ providedȱ byȱ theȱ Stateȱ throughȱ theȱ ATCȱ annualȱ budget.ȱ Onceȱ again,ȱtheȱtransitionalȱarrangementsȱworkedȱwellȱandȱnowȱallȱILSAȱcostsȱ areȱmetȱthroughȱATC’sȱoperationalȱbudget.ȱ DriversȱofȱChangeȱ Initially,ȱ theȱ fundamentalȱ driverȱ ofȱ changeȱ wasȱ GORA’sȱ commitmentȱ toȱ theȱ wholesaleȱ restructuringȱ andȱ reformȱ ofȱ theȱ legacyȱ Sovietȱ systemȱ toȱ ensureȱ thatȱ nationalȱ educationalȱ outcomesȱ wouldȱ beȱ appropriateȱ forȱ aȱ modernȱcountryȱcompetingȱinȱaȱglobalȱmarket.ȱItȱwasȱrecognizedȱthatȱthisȱ wouldȱ requireȱ anȱ internationalȱ perspectiveȱ andȱ comparativeȱ dataȱ onȱ educationalȱstandardsȱandȱtrends.ȱAnȱexternalȱdriverȱthatȱresonatedȱwithȱ GORA’sȱobjectivesȱwasȱprovidedȱbyȱtheȱWorldȱBankȱwhich,ȱatȱtheȱtimeȱofȱ planningȱ forȱ EQRPȱ I,ȱ wasȱ increasinglyȱ advocatingȱ thatȱ countriesȱ investingȱ inȱ educationȱ shouldȱ participateȱ inȱ ILSAȱ inȱ orderȱ toȱ gainȱ evidenceȱ toȱ informȱ policyȱ making.ȱ TIMSSȱ was,ȱ atȱ theȱ time,ȱ theȱ predominantȱ ILSAȱ andȱ beingȱ curriculumȬbasedȱ wasȱ theȱ obviousȱ choiceȱ forȱArmenia.ȱ Theȱ developmentȱ ofȱ capacityȱ withinȱ ATCȱ underȱ EQRPȱ I,ȱ andȱ throughȱ participationȱ inȱ TIMSS,ȱ contributedȱ toȱ theȱ establishmentȱ ofȱ aȱ professionalȱ assessmentȱ communityȱ where,ȱ toȱ allȱ intentsȱ andȱ purposes,ȱ noneȱ existedȱ before.ȱ Similarly,ȱ theȱ involvementȱ ofȱ theȱ MOESȱ inȱ theȱ projectȱ generatedȱincreasedȱ interestȱ inȱ theȱ useȱ ofȱ assessmentȬrelatedȱ dataȱ forȱ systemȱ monitoring.ȱ Theȱ combinationȱ ofȱ theȱ twoȱ servedȱ asȱ anȱ internalȱ driverȱ forȱ theȱ developmentȱ ofȱ theȱ twoȱ formsȱ ofȱ NLSAȱ currentlyȱ beingȱ implementedȱbyȱtheȱATC.ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 31ȱ DiscussionȱandȱLessonsȱLearnedȱ SummaryȱofȱProgressȱMadeȱ Overȱ aȱ periodȱ ofȱ littleȱ moreȱ thanȱ twoȱ decades,ȱ Armeniaȱ hasȱ madeȱ greatȱ progressȱ inȱ itsȱ effortsȱ toȱ developȱ aȱ comprehensiveȱ systemȱ ofȱ studentȱ assessmentȱ toȱ monitorȱ andȱ enhanceȱ educationalȱ outcomesȱ acrossȱ theȱ generalȱ phaseȱ ofȱ education.ȱ Advancesȱ haveȱ beenȱ madeȱ onȱ allȱ fronts:ȱ theȱ universityȱadmissionsȱprocessȱhasȱbeenȱmadeȱfairerȱandȱmoreȱtransparentȱ throughȱ theȱ introductionȱ ofȱ theȱ UE;ȱ teachingȱ andȱ learningȱ haveȱ beenȱ enhancedȱ throughȱ theȱ introductionȱ ofȱ betterȱ classroomȱ assessmentȱ practices;ȱ andȱ nationalȱ andȱ internationalȱ sampleȬbasedȱ assessmentsȱ areȱ startingȱtoȱprovideȱpotentiallyȱvaluableȱinformationȱtoȱeducationalȱpolicyȱ makersȱ andȱ practitioners.ȱ Theȱ ATCȱ nowȱ servesȱ asȱ aȱ sustainableȱ nationalȱ centerȱ ofȱ excellenceȱ forȱ assessmentȱ activities,ȱ andȱ aroundȱ it,ȱ aȱ professionalȱ assessmentȱ communityȱ isȱ emergingȱ withȱ theȱ potentialȱ toȱ maintainȱ andȱ furtherȱ strengthenȱ assessmentȱ practices.ȱ Muchȱ remainsȱ toȱ beȱdoneȱinȱallȱofȱtheseȱareas,ȱbutȱtheȱfoundationsȱareȱfirm.ȱ KeyȱDriversȱ Theȱ driversȱ behindȱ Armenia’sȱ progressȱ inȱ thisȱ fieldȱ haveȱ beenȱ bothȱ externalȱ andȱ internal.ȱ Externally,ȱ agenciesȱ suchȱ asȱ theȱ Worldȱ Bankȱ haveȱ consistentlyȱreinforcedȱtheȱmessage,ȱbasedȱonȱinternationalȱevidence,ȱthatȱ forȱ aȱ modernȱ educationȱ systemȱ toȱ beȱ effectiveȱ andȱ efficient,ȱ aȱ wellȬdesignedȱstudentȱassessmentȱsystemȱisȱneededȱtoȱcomplementȱallȱofȱ theȱ otherȱ necessaryȱ elementsȱ ofȱ theȱ teaching/learningȱ process.ȱ Thisȱ messageȱ wasȱ wellȱ receivedȱ byȱ GORA,ȱ whichȱ proposedȱ thatȱ creditȱ agreementsȱ forȱ educationalȱ reformȱ projectsȱ shouldȱ includeȱ significantȱ fundsȱ explicitlyȱ forȱ theȱ enhancementȱ ofȱ educationalȱ assessment.ȱ Inȱ Armenia,ȱ projects—mostȱ notablyȱ EQRPȱ I—haveȱ providedȱ resourcesȱ andȱ aȱ structureȱ forȱ drivingȱ forwardȱ assessmentȬrelatedȱ reforms.ȱ However,ȱ externalȱ driversȱ onȱ theirȱ ownȱ areȱ neverȱ sufficient.ȱ Indeed,ȱ itȱ isȱ undoubtedlyȱ theȱ internalȱ driversȱ thatȱ ultimatelyȱ determineȱ theȱ directionȱ ofȱ travelȱ andȱ theȱ rateȱ ofȱ progress.ȱ Inȱ Armenia,ȱ theȱ mostȱ powerfulȱ driverȱ wasȱ andȱ remainsȱ theȱ government’sȱ commitmentȱ toȱ improvingȱ educationȱ inȱ orderȱ toȱ strengthenȱ theȱ country’sȱ mostȱ valuableȱ economicȱ resource—itsȱ people.ȱ Thisȱ canȱ beȱ seenȱ inȱ itsȱ willingnessȱ toȱ investȱ heavilyȱ ȱ 32ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ notȱ justȱ inȱ infrastructure,ȱ butȱ alsoȱ inȱ theȱ ‘softer’ȱ areasȱ ofȱ education,ȱ suchȱ asȱ theȱ developmentȱ ofȱ modernȱ subjectȱ curricula,ȱ learningȱ standards,ȱ andȱ assessmentȱpractices.ȱ ȱ Aȱ secondaryȱ internalȱ driver,ȱ butȱ oneȱ withȱ aȱ veryȱ highȱ profile,ȱ wasȱ theȱ government’sȱ commitmentȱ toȱ reducingȱ theȱ perceivedȱ levelȱ ofȱ corruptionȱ inȱ assessmentȱ practices.ȱ Thisȱ hasȱ beenȱ anȱ objectiveȱ sharedȱ byȱ manyȱ formerȱ socialistȱ republics,ȱ andȱ theȱ firstȱ stepȱ isȱ almostȱ alwaysȱ toȱ reformȱ traditionalȱ universityȱ admissionsȱ proceduresȱ throughȱ theȱ introductionȱ ofȱ aȱ secure,ȱ centralizedȱ examinationȱ system.ȱ Inevitably,ȱ thisȱ hasȱ bothȱ positiveȱ andȱ negativeȱ consequences,ȱ althoughȱ theȱ immediateȱ benefitsȱforȱsocietyȱ(e.g.,ȱgreaterȱequityȱinȱaccessȱtoȱHEI)ȱusuallyȱoutweighȱ theȱ disadvantages.ȱ Inȱ Armenia,ȱ theȱ driveȱ toȱ reduceȱ corruptionȱ throughȱ theȱ developmentȱ ofȱ theȱ UEȱ ledȱ indirectlyȱ toȱ theȱ establishmentȱ ofȱ theȱ ATC—aȱkeyȱmechanismȱinȱtheȱreformȱofȱgeneralȱassessmentȱpractices.ȱ Aȱ thirdȱ internalȱ driverȱ wasȱ theȱ Government’sȱ desireȱ toȱ alignȱ assessmentȱpracticesȱmoreȱcloselyȱwithȱinternationalȱbestȱpractice.ȱInȱthis,ȱ itȱ sawȱ theȱ potentialȱ toȱ increaseȱ fairness,ȱ transparency,ȱ andȱ objectivityȱ inȱ decisionȱ makingȱ throughȱ theȱ useȱ ofȱ modernȱ technologies.ȱ Itȱ alsoȱ recognizedȱ theȱ needȱ toȱ developȱ aȱ cadreȱ ofȱ assessmentȱ professionalsȱ inȱ orderȱtoȱimplementȱandȱsustainȱtheȱdesiredȱreforms.ȱ ȱ Inȱ theȱ caseȱ ofȱ Armenia,ȱ clearlyȬarticulatedȱ goalsȱ andȱ objectivesȱ resonatedȱ stronglyȱ withȱ externalȱ drivers,ȱ givingȱ greaterȱ impetusȱ toȱ reformȱinitiativesȱandȱincreasingȱtheȱchancesȱofȱsuccess.ȱ ȱ KeyȱMechanismsȱforȱBringingȱAboutȱReformȱ Theȱ mostȱ effectiveȱ mechanismȱ forȱ bringingȱ aboutȱ reformȱ inȱ theȱ areaȱ ofȱ studentȱ assessmentȱ hasȱ undoubtedlyȱ beenȱ theȱ GORA’sȱ judiciousȱ useȱ ofȱ educationalȱ projectsȱ toȱ implementȱ initiativesȱ inȱ aȱ rangeȱ ofȱ wellȬdefinedȱ areas:ȱhighȬstakeȱexaminations,ȱclassroomȱassessment,ȱteacherȱtrainingȱinȱ assessment,ȱ andȱ participationȱ inȱ internationalȱ studiesȱ (TIMSS).ȱ Fundsȱ madeȱ availableȱ throughȱ creditȱ agreementsȱ enabledȱ GORAȱ toȱ supportȱ capitalȱ expenditureȱ in,ȱ forȱ example,ȱ establishingȱ theȱ ATCȱ asȱ aȱ fullyȬoperationalȱ organizationȱ atȱ aȱ timeȱ whenȱ theȱ nationalȱ budgetȱ wasȱ underȱ greatȱ pressureȱ fromȱ manyȱ competingȱ interests.ȱ Subsequently,ȱ operationalȱ costsȱ associatedȱ withȱ educationalȱ assessmentȱ haveȱ beenȱ transferredȱ toȱ appropriateȱ stateȱ budgetȱ lines,ȱ ensuringȱ longȬtermȱ sustainability.ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 33ȱ Maximizingȱ theȱ effectivenessȱ ofȱ suchȱ projectsȱ requiresȱ carefulȱ planningȱ priorȱ toȱ implementationȱ andȱ goodȱ managementȱ duringȱ implementation.ȱ Theȱ mechanismȱ forȱ ensuringȱ thatȱ bothȱ aspectsȱ wereȱ adequatelyȱ addressedȱ wasȱ throughȱ theȱ creationȱ ofȱ aȱ designatedȱ projectȱ implementationȱ unitȱ (i.e.,ȱ theȱ CFEP)ȱ inȱ theȱ MOES.ȱ Theȱ responsibilitiesȱ ofȱ theȱ CFEPȱ includedȱ notȱ onlyȱ theȱ administrativeȱ andȱ financialȱ elementsȱ ofȱ projectȱ management,ȱ butȱ alsoȱ monitoringȱ theȱ qualityȱ ofȱ professionalȱ servicesȱandȱactivitiesȱandȱtheirȱalignmentȱwithȱMOESȱexpectations.ȱ Internationalȱ consultantsȱ were,ȱ andȱ continueȱ toȱ be,ȱ aȱ majorȱ mechanismȱ inȱ implementingȱ assessmentȱ reformsȱ inȱ Armenia.ȱ Someȱ countriesȱ areȱ reluctantȱ toȱ useȱ internationalȱ expertiseȱ becauseȱ ofȱ theȱ difficultyȱofȱensuringȱvalueȱforȱmoney.ȱInȱArmenia,ȱthereȱisȱaȱwillingnessȱ toȱuseȱinternationalȱtechnicalȱassistanceȱprovidedȱitȱisȱofȱhighȱqualityȱandȱ providedȱthatȱlocalȱcontrolȱcanȱbeȱmaintained.ȱ Anotherȱprominentȱmechanismȱinȱtransferringȱknowledgeȱandȱskillsȱ hasȱ beenȱ theȱ significantȱ investmentȱ inȱ trainingȱ acrossȱ aȱ numberȱ ofȱ assessmentȬrelatedȱ areasȱ andȱ forȱ aȱ wideȱ rangeȱ ofȱ beneficiaries.ȱ Trainingȱ hasȱ beenȱ providedȱ bothȱ withȱ internationalȱ technicalȱ assistanceȱ andȱ throughȱ localȱ providersȱ suchȱ asȱ theȱ NIE.ȱ Underȱ EQRPȱ I,ȱ oneȱ strandȱ wasȱ toȱ buildȱ professionalȱ capacityȱ throughȱ trainingȱ ATCȱ staffȱ andȱ associateȱ testȱ developersȱ (mainlyȱ fromȱ universities)ȱ inȱ theȱ principlesȱ andȱ practicesȱ ofȱ assessmentȱ andȱ measurement.ȱ Aȱ second,ȱ complementaryȱ strandȱ wasȱ theȱ provisionȱ ofȱ trainingȱ inȱ classroomȱ assessmentȱ techniques.ȱ Here,ȱ leadȱ trainersȱpreparedȱunderȱtheȱprojectȱprovidedȱlocal,ȱschoolȬbasedȱtrainingȱ whichȱ reached,ȱ toȱ allȱ intentsȱ andȱ purposes,ȱ everyȱ teacher.ȱ Thisȱ commitmentȱ toȱ trainingȱ continuesȱ withȱ theȱ furtherȱ strengtheningȱ ofȱ inȬserviceȱ professionalȱ developmentȱ forȱ teachersȱ andȱ theȱ preparationȱ ofȱ universityȱstaffȱtoȱdeliverȱtheȱpreȬserviceȱmoduleȱonȱassessment.ȱ ȱ Finally,ȱ theȱ ATCȱ establishedȱ underȱ EQRPȱ Iȱ nowȱ servesȱ asȱ aȱ majorȱ mechanismȱ inȱ implementingȱ andȱ sustainingȱ reformsȱ inȱ theȱ assessmentȱ system.ȱ GORAȱ hasȱ investedȱ heavilyȱ inȱ ATC’sȱ accommodation,ȱ equipment,ȱ andȱ staffȱ development.ȱ Itȱ continuesȱ toȱ fundȱ theȱ organizationȱ fromȱ aȱ dedicatedȱ lineȱ inȱ theȱ stateȱ budget.ȱ Inȱ return,ȱ ATCȱ nowȱ offersȱ aȱ naturalȱ homeȱ forȱ allȱ theȱ examinationȱ andȱ assessmentȱ activitiesȱ currentlyȱ requiredȱbyȱtheȱMOESȱandȱitȱhasȱtheȱcapacityȱtoȱtakeȱonȱnewȱinitiativesȱinȱ theȱ future.ȱ Theȱ effectivenessȱ ofȱ thisȱ hasȱ alreadyȱ beenȱ shownȱ throughȱ theȱ gradualȱ transferȱ ofȱ additionalȱ responsibilitiesȱ toȱ ATCȱ afterȱ theȱsuccessfulȱ ȱ 34ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ introductionȱ ofȱ theȱ UE,ȱ includingȱ responsibilityȱ forȱ internationalȱ andȱ nationalȱlargeȬscaleȱsurveysȱofȱlearnerȱachievement.ȱ ȱ SelectedȱLessonsȱLearnedȱandȱAssociatedȱChallengesȱ Lessonȱ1:ȱTheȱtechnicalȱelementsȱofȱnewȱformsȱofȱassessmentȱareȱrelativelyȱeasyȱtoȱ introduce;ȱ itȱ isȱ farȱ moreȱ difficultȱ toȱ ensureȱ thatȱ theȱ informationȱ theyȱ generateȱ isȱ usedȱeffectivelyȱbyȱpolicyȱmakersȱandȱpractitioners.ȱ AtȱtheȱstartȱofȱArmenia’sȱeducationalȱreforms,ȱtheȱcapacityȱtoȱdesignȱandȱ implementȱ highȬqualityȱ examinationsȱ andȱ sampleȬbasedȱ surveysȱ ofȱ learnerȱ achievementȱ wasȱ extremelyȱ limited.ȱ However,ȱ throughȱ theȱ mechanismsȱ describedȱ inȱ thisȱ paper,ȱ theȱ countryȱ has,ȱ amongstȱ otherȱ things,ȱ successfullyȱ introducedȱ aȱ highȬstakesȱ examinationȱ forȱ universityȱ selectionȱandȱconductedȱthreeȱcyclesȱofȱanȱILSA.ȱArmeniaȱisȱnotȱuniqueȱinȱ this.ȱ Sinceȱ 1991,ȱ practicallyȱ allȱ formerȱ Sovietȱ republicsȱ haveȱ establishedȱ centralizedȱ examinationȱ systemsȱ fromȱ scratchȱ (Bethellȱ andȱ Zabulionis,ȱ 2012).ȱ Theȱ testȱ papersȱ mayȱ notȱ alwaysȱ beȱ ofȱ theȱ highestȱ quality,ȱ andȱ oneȱ mayȱ questionȱ theȱ educationalȱ validityȱ ofȱ someȱ ofȱ theȱ proceduresȱ used,ȱ butȱ fromȱ aȱ technicalȱ pointȱ ofȱ viewȱ theȱ newȱ systemsȱ areȱ operationalȱ and,ȱ inȱ general,ȱ haveȱ fewȱ majorȱ problems.ȱ Inȱ addition,ȱ manyȱ formerȱ Sovietȱ republicsȱ haveȱ participatedȱ inȱ atȱ leastȱ oneȱ majorȱ ILSAȱ and,ȱ withȱ aȱ fewȱ exceptions,ȱ haveȱ successfullyȱ metȱ internationalȱ proceduralȱ standards.ȱ Theȱ conclusionȱ isȱ clear:ȱ withȱ sufficientȱ investmentȱ andȱ preparation,ȱ countriesȱ doȱ notȱ findȱ itȱ difficultȱ toȱ buildȱ theȱ technicalȱ capacityȱ toȱ ‘do’ȱ anȱ assessment.ȱ However,ȱ examinationsȱ andȱ otherȱ formsȱ ofȱ assessmentȱ areȱ conductedȱ inȱ orderȱ toȱ serveȱ specificȱ purposes,ȱ includingȱ theȱ provisionȱ ofȱ potentiallyȱvaluableȱinformationȱforȱinformingȱtheȱdecisionsȱofȱpolicyȱmakersȱandȱ guidingȱ teachersȱ andȱ otherȱ practitioners.ȱ Itȱ isȱ thisȱ elementȱ thatȱ isȱ oftenȱ neglectedȱorȱbadlyȱimplemented.ȱ Inȱ Armenia,ȱ theȱ stateȱ investsȱ aȱ significantȱ sumȱ fromȱ itsȱ budgetȱ inȱ orderȱ toȱ participateȱ inȱ TIMSS.ȱ Theȱ country’sȱ resultsȱ areȱ presentedȱ inȱ theȱ internationalȱreport,ȱallowingȱsomeȱbasicȱcomparisonsȱtoȱbeȱmade.ȱThereȱ isȱ alsoȱ aȱ preliminaryȱ nationalȱ reportȱ thatȱ summarizesȱ studentȱ performanceȱ basedȱ onȱ rawȱ scores,ȱ butȱ itȱ isȱ notȱ clearȱ whatȱ valueȱ thisȱ has.ȱ Noȱ furtherȱ secondaryȱ analysisȱ isȱ conducted,ȱ noȱ independentȱ researchȱ isȱ commissioned,ȱ andȱ noȱ subjectȬspecificȱ reportsȱ areȱ preparedȱ forȱ schoolsȱ andȱteachers.ȱAsȱaȱconsequence,ȱmuchȱpotentiallyȱvaluableȱinformationȱisȱ leftȱunused.ȱThisȱisȱnotȱunusual,ȱasȱevidencedȱbyȱtheȱfollowingȱstatementȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 35ȱ fromȱ anȱ OECDȱ Educationȱ Workingȱ Paper:ȱ “inȱ mostȱ lowȬȱ andȱ middleȬincomeȱ countries,ȱ thereȱ isȱ noȱ PISAȱ nationalȱ reportȱ andȱ noȱ nationalȱ dataȱ analysis.ȱ Reasonsȱ vary,ȱ fromȱ lowȱ analyticalȱ capacityȱ toȱ humanȱ andȱ financialȱ resourceȱ restrictions,ȱ orȱ lackȱ ofȱ politicalȱ interest.”ȱ (Bloem,ȱ2013,ȱp.22).ȱ Theȱ OECDȱ statementȱ revealsȱ twoȱ criticalȱ dimensionsȱ inȱ establishingȱ anȱ effectiveȱ feedbackȱ system:ȱ politicalȱ interestȱ andȱ analyticalȱ capacity.ȱ Theȱfirstȱofȱtheseȱgeneratesȱtheȱdemandȱforȱinformationȱand,ȱinȱparticular,ȱ relevantȱ informationȱ presentedȱ inȱ anȱ appropriateȱ form.ȱ Theȱ secondȱ suppliesȱ theȱ informationȱ demandedȱ inȱ theȱ requiredȱ form.ȱ Theȱ simplestȱ scenarioȱ isȱ uniȬdirectional,ȱ i.e.,ȱ theȱ MOESȱ asksȱ ATCȱ toȱ supplyȱ relevantȱ informationȱ andȱ ATCȱ conductsȱ theȱ analysisȱ toȱ provideȱ it.ȱ However,ȱ thisȱ willȱ notȱ workȱ ifȱ policyȱ makersȱ doȱ notȱ knowȱ whatȱ toȱ askȱ for,ȱ perhapsȱ becauseȱ theyȱ doȱ notȱ knowȱ whatȱ isȱ availableȱ orȱ howȱ theyȱ canȱ useȱ suchȱ data.ȱ Therefore,ȱ inȱ practice,ȱ theȱ relationshipȱ shouldȱ beȱ biȬdirectional:ȱ assessmentȱ specialistsȱ withinȱ ATCȱ shouldȱ generateȱ additionalȱ demandȱ byȱprovidingȱpolicyȱmakersȱwithȱtheȱrichestȱsetȱofȱdataȱpossibleȱpresentedȱ inȱuserȬfriendlyȱformats.ȱ Onceȱ policyȱ makersȱseeȱ theȱpotentialȱ benefitsȱ ofȱ assessmentȬrelatedȱ data,ȱ itȱ isȱ likelyȱ thatȱ theyȱ willȱ askȱ forȱ more,ȱ therebyȱ reinforcingȱ theȱ feedbackȱ loop.ȱ Achievingȱ thisȱ requiresȱ actionȱ onȱ threeȱ fronts:ȱ ȱ x educatingȱ policyȱ makersȱ soȱ thatȱ theyȱ canȱ interpretȱ assessmentȬrelatedȱdataȱintelligently;ȱ x motivatingȱassessmentȱspecialistsȱsoȱthatȱtheyȱgenerateȱmoreȱandȱ betterȱdata;ȱand,ȱ x providingȱ resourcesȱ soȱ thatȱ secondaryȱ analysisȱ andȱ focusedȱ researchȱcanȱbeȱconducted.ȱ ȱ Inȱ additionȱ toȱ providingȱ dataȱ toȱ policyȱ makers,ȱ theȱ assessmentȱ agencyȱ (ATCȱ inȱ theȱ caseȱ ofȱ Armenia)ȱ shouldȱ provideȱ feedbackȱ toȱ subjectȱ teachersȱsoȱthatȱtheyȱcanȱbetterȱunderstandȱtheȱstrengthsȱandȱweaknessesȱ ofȱ typicalȱ learnersȱ andȱ henceȱ improveȱ theirȱ teachingȱ practices.ȱ Forȱ teachers,ȱ rawȱ scoresȱ andȱ otherȱ descriptiveȱ statisticsȱ presentedȱ outȱ ofȱ contextȱ areȱ ofȱ little,ȱ ifȱ any,ȱ value.ȱ Therefore,ȱ subjectȱ specialistsȱ withȱ anȱ understandingȱ ofȱ assessmentȱ statisticsȱ needȱ toȱ selectȱ andȱ presentȱ dataȱ inȱ formsȱ thatȱ teachersȱ canȱ use.ȱ Forȱ example,ȱ TIMSSȱ internationalȱ reportsȱ includeȱaȱlimitedȱnumberȱofȱexemplarȱitemsȱforȱMathematicsȱandȱScienceȱ ȱ 36ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ withȱ dataȱ onȱ theȱ proportionȱ ofȱ studentsȱ respondingȱ correctlyȱ inȱ eachȱ participatingȱcountry.ȱ Examplesȱ ofȱ goodȱ practiceȱ inȱ nationalȱ sampleȬbasedȱ assessmentsȱ includeȱ theȱ comprehensiveȱ subjectȱ reportsȱ producedȱ fromȱ dataȱ gatheredȱ inȱtheȱNationalȱAssessmentȱofȱEducationalȱProgressȱ(NAEP)ȱconductedȱinȱ theȱUSA.ȱTheseȱincludeȱ‘itemȱmaps’ȱthatȱsetȱout,ȱusingȱconcreteȱexamples,ȱ whichȱ tasksȱ haveȱ beenȱ masteredȱ byȱ studentsȱ atȱ differentȱ partsȱ ofȱ theȱ abilityȱrange.29ȱ ȱ Inȱ thisȱ feedbackȱ loop,ȱ theȱ communityȱ ofȱ teachersȱ isȱ unlikelyȱ toȱ initiateȱ demand.ȱ Therefore,ȱ theȱ assessmentȱ agencyȱ needsȱ toȱ beȱ proactiveȱ andȱ assumeȱ responsibilityȱ forȱ providingȱ usefulȱ informationȱ inȱ appropriateȱ formats.ȱ Itȱ alsoȱ needsȱ toȱ contribute,ȱ perhapsȱ throughȱ collaborationȱ withȱ otherȱ organizations,ȱ toȱ theȱ trainingȱ ofȱ teachersȱ inȱ interpretingȱassessmentȬrelatedȱdata.ȱ ȱ Lessonȱ2:ȱInȱtheȱcaseȱofȱexaminations,ȱwhereȱtheȱstakesȱareȱhigh,ȱthereȱisȱinevitablyȱ aȱ tensionȱ betweenȱ theȱ needȱ toȱ maintainȱ examinationȱ securityȱ andȱ theȱ desireȱ toȱ promoteȱ importantȱ educationalȱ objectivesȱ throughȱ moreȱ innovativeȱ andȱ effectiveȱ approachesȱtoȱexaminationȱdesign.ȱ Armenia,ȱ inȱ commonȱ withȱ manyȱ otherȱ postȬsocialistȱ states,ȱ introducedȱ centralizedȱ examinationsȱ forȱ universityȱ selectionȱ purposesȱ inȱ aȱ climateȱ whereȱtheȱperceptionȱofȱcorruptionȱisȱgreat.30ȱ Theȱpressingȱneedȱtoȱmakeȱ theȱ newȱ systemȱ objective,ȱ transparent,ȱ andȱ secureȱ dictatedȱ theȱ useȱ ofȱ soȬcalledȱ ‘technological’ȱ itemsȱ capableȱ ofȱ beingȱ scoredȱ byȱ computerizedȱ systems.ȱ Theȱ obsessionȱ withȱ transparencyȱ alsoȱ ledȱ toȱ theȱ adoptionȱ ofȱ anȱ overlyȬsimpleȱscoringȱandȱreportingȱscale,ȱasȱpreviouslyȱdescribed.ȱTheseȱ measures,ȱ coupledȱ withȱ theȱ useȱ ofȱ highlyȱ visible,ȱ antiȬcheatingȱ measuresȱ inȱtestingȱcentersȱensuredȱthatȱArmenia’sȱUEȱwasȱsuccessfullyȱintroducedȱ inȱ ‘safeȱ mode.’ȱ However,ȱ placingȱ theȱ emphasisȱ onȱ antiȬcorruptionȱ inevitablyȱ relegatedȱ certainȱ educationalȱ objectives,ȱ e.g.,ȱ promotingȱ theȱ teachingȱandȱlearningȱofȱhigherȬlevelȱcognitiveȱskills,ȱtoȱsecondȱplace.ȱTheȱ presumptionȱ wasȱ thatȱ onceȱ theȱ newȱ systemȱ hadȱ embedded,ȱ moreȱ validȱ formsȱofȱassessment,ȱsuchȱasȱtheȱuseȱofȱproblemȬsolvingȱtasksȱsetȱinȱnovelȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 29ȱ SubjectȱreportsȱforȱNAEPȱcanȱbeȱfoundȱatȱhttp://nces.ed.gov/nationsreportcard/ȱ(accessedȱ 13ȱJulyȱ2015).ȱ 30ȱ Transparencyȱ International’sȱ Corruptionsȱ Perceptionȱ Indexȱ forȱ 2013ȱ placesȱ Armeniaȱ atȱ aȱ rankȱofȱ94ȱoutȱofȱ175ȱcountriesȱ(TransparencyȱInternational,ȱ2013).ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 37ȱ contextsȱ andȱ extendedȬresponseȱ itemsȱ (writtenȱ components)ȱ couldȱ beȱ introduced.ȱInȱreality,ȱtheȱconcernȱwithȱsecurityȱandȱtransparencyȱhasȱnotȱ receded.ȱ Onȱ theȱ contrary,ȱ recentȱ policyȱ interventions—inȱ particularȱ theȱ moveȱ towardsȱ theȱ useȱ ofȱ openȱ itemȱ banks—haveȱ shiftedȱ theȱ systemȱ furtherȱ awayȱ fromȱ highlyȱ desirableȱ educationalȱ valuesȱ towardsȱ mechanisticȱ transparencyȱ andȱ antiȬcorruptionȱ measures.ȱ Thereȱ areȱ undoubtedlyȱ someȱ shortȬtermȱ gainsȱ with,ȱ forȱ example,ȱ fewerȱ complaintsȱ beingȱ lodgedȱ aboutȱ theȱ examinationȱ questions.ȱ However,ȱ whatȱ willȱ beȱ theȱ longȬtermȱ impactȱ onȱ nationalȱ educationalȱ outcomes?ȱ Willȱ theȱ youngȱ menȱ andȱ womenȱ whoȱ emergeȱ fromȱ anȱ educationȱ systemȱ dominatedȱ byȱ roteȱlearningȱhaveȱtheȱhigherȬlevelȱskillsȱrequiredȱbyȱaȱmodernȱworkplaceȱ inȱanȱincreasinglyȱcompetitiveȱglobalȱmarket?ȱ ȱ Thisȱ tensionȱ betweenȱ maintainingȱ securityȱ andȱ harnessingȱ theȱ potentialȱpositiveȱbackwashȱeffectȱofȱhighȬstakeȱexaminationsȱwillȱpersistȱ whileȱ society’sȱ perceptionȱ ofȱ corruptionȱ remainsȱ high.ȱ Thereȱ areȱ noȱ simpleȱ solutions.ȱ However,ȱ policyȱ makersȱ shouldȱ beȱ awareȱ thatȱ anȱ appropriateȱbalanceȱisȱrequiredȱifȱdesiredȱeducationalȱoutcomesȱareȱtoȱbeȱ promoted.ȱ Furthermore,ȱ itȱ isȱ theȱ roleȱ ofȱ theȱ assessmentȱ communityȱ toȱ conveyȱthisȱmessageȱandȱtoȱproposeȱsuitableȱstrategies.ȱ Lessonȱ 3:ȱ Assessmentsȱ areȱ notȱ isolatedȱ fromȱ otherȱ aspectsȱ ofȱ society.ȱ Asȱ theȱ contextȱinȱwhichȱtheyȱoperateȱchanges,ȱthereȱwillȱbeȱaȱneedȱtoȱreviewȱtheirȱ‘fitnessȱ forȱpurpose’ȱand,ȱwhereȱnecessary,ȱadaptȱthemȱaccordingly.ȱ ȱ Whenȱ evaluatingȱ studentȱ assessmentȱ systems,ȱ itȱ isȱ allȱ tooȱ easyȱ toȱ getȱ caughtȱ upȱ inȱtheȱ technicalȱaspectsȱandȱ toȱ forgetȱ thatȱassessmentȱisȱ notȱ anȱ endȱ inȱ itsȱ ownȱ right—itȱ isȱ merelyȱ aȱ toolȱ thatȱ shouldȱ serveȱ theȱ educationȱ systemȱ andȱ societyȱ asȱ aȱ whole.ȱ Inȱ thisȱ subservientȱ role,ȱ assessmentȱ systemsȱ shouldȱ beȱ dynamicȱ andȱ respondȱ toȱ changingȱ socioȬeconomicȱ contextsȱ andȱ shiftingȱ priorities.ȱ Armenia’sȱ UEȱ providesȱ anȱ interestingȱ exampleȱofȱthis.ȱ Universityȱ selectionȱ examinationsȱ areȱ aȱ mechanismȱ byȱ whichȱ demandȱforȱfurtherȱeducationȱisȱmatchedȱwithȱsupply.ȱWhenȱtheȱUEȱwasȱ introducedȱinȱ2007/2008,ȱcompetitionȱforȱplacesȱinȱpopularȱcoursesȱinȱHEIȱ wasȱ significant,ȱ especiallyȱ forȱ stateȬfundedȱ places.ȱ Whereȱ universityȱ facultiesȱ facedȱ theȱ problemȱ ofȱ numerusȱ clausus,ȱ scoresȱ fromȱ UEȱ subjectȱ testsȱ providedȱ objectiveȱ informationȱ forȱ allocatingȱ bothȱ placesȱ andȱ anyȱ availableȱscholarships.ȱSinceȱthen,ȱhowever,ȱtheȱnation’sȱdemographyȱhasȱ ȱ 38ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ changedȱ significantlyȱ dueȱ toȱ aȱ numberȱ ofȱ factors,ȱ includingȱ fallingȱ birthȱ ratesȱ andȱ increasingȱ migration.ȱ Forȱ example,ȱ totalȱ enrolmentȱ forȱ theȱ generalȱ phaseȱ ofȱ educationȱ inȱ stateȱ andȱ privateȱ schoolsȱ hasȱ fallenȱ fromȱ 411,439ȱ inȱ 2010/2011ȱ toȱ 359,966ȱ inȱ 2013/2014—aȱ dropȱ ofȱ 13ȱ percentȱ (NationalȱCenterȱofȱEducationalȱTechnologiesȱ[NCET],ȱ2015).ȱTheȱchangeȱ isȱ evenȱ moreȱ startlingȱ whenȱ enrolmentȱ inȱ theȱ yearȱ ofȱ graduationȱ isȱ consideredȱ with,ȱ asȱ shownȱ inȱ Figureȱ 2,ȱ aȱ dropȱ ofȱ moreȱ thanȱ 36ȱ percentȱ sinceȱ2010/2011.31ȱ ȱ Figureȱ2:ȱAnnualȱenrolmentȱdataȱforȱtheȱfinalȱyearȱofȱgeneralȱsecondaryȱ educationȱ(G11ȱinȱ2010/2011ȱandȱG12ȱthereafter).ȱ ȱ 45000  100% 40000 93.8% 35000 74.4% 30000 63.4% # enrolled 25000 20000 15000 10000 5000 0 2010/11 2011/12 2012/13 2013/14 ȱ Source:ȱNationalȱCenterȱforȱEducationalȱTechnologiesȱatȱhttp://stat.armedu.am/.ȱ ȱ Withȱ theȱ schoolȱ populationȱ fallingȱ rapidly,ȱ theȱabsoluteȱ demandȱforȱ universityȱ placesȱ fell,ȱ butȱ theȱ numberȱ ofȱ placesȱ wasȱ notȱ reducedȱ accordingly.ȱ Withȱ relativeȱ supplyȱ increasing,ȱ competitionȱ wasȱ lessȱ fierceȱ andȱ theȱ perceivedȱ needȱ forȱ theȱ selectionȱ examinationȱ wasȱ reduced.ȱ Toȱ complicateȱtheȱmatterȱfurther,ȱthisȱdemographicȱshiftȱtookȱplaceȱagainstȱaȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 31ȱ Inȱ theȱ yearȱ 2010/2011,ȱ studentsȱ graduatedȱ inȱ Gradeȱ 11.ȱ Sinceȱ thenȱ studentsȱ haveȱ graduatedȱinȱGradeȱ12.ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 39ȱ backdropȱ ofȱ decliningȱ economicȱ strength,ȱ whichȱ causedȱ theȱ governmentȱ toȱ reduceȱ fundingȱ forȱ HEIȱ byȱ cuttingȱ theȱ numberȱ ofȱ scholarships,ȱ especiallyȱ forȱ popularȱ specialtiesȱ suchȱ asȱ Economicsȱ andȱ Lawȱ whereȱ demandȱ wasȱ stillȱ great.ȱ Universitiesȱ responded,ȱ notȱ byȱ cuttingȱ costsȱ byȱ closingȱ underȬsubscribedȱ courses,ȱ butȱ byȱ increasingȱ theȱ numberȱ ofȱ placesȱ forȱ feeȬpayingȱ studentsȱ onȱ popularȱ coursesȱ inȱ orderȱ toȱ maintainȱ income.ȱ Thisȱ increasedȱ theȱ supplyȱ inȱ absoluteȱ termsȱ which,ȱ inȱ turn,ȱ reducedȱ competitionȱstillȱfurther.ȱ Fromȱ 2007ȱ toȱ theȱ present,ȱ HEIȱ enrolmentȱ ratesȱ inȱ Armeniaȱ haveȱ increased.ȱ Thisȱ mayȱ beȱ aȱ desirableȱ outcomeȱ forȱ aȱ countryȱ withȱ aȱ veryȱ largeȱ cohort,ȱ butȱ inȱ Armenia,ȱ admittingȱ aȱ largerȱ proportionȱ ofȱ studentsȱ meansȱ thatȱ theȱ averageȱ qualityȱ ofȱ theȱ intakeȱ toȱ HEIȱ isȱ lowerȱ thanȱ inȱ previousȱ years.ȱ Theȱ impactȱ isȱ mostȱ noticeableȱ inȱ lessȱ popular,ȱ butȱ nationallyȱ vital,ȱ coursesȱ suchȱ asȱ Agriculture,ȱ Scienceȱ (Biologyȱ andȱ Chemistry),ȱandȱEngineeringȱwhereȱstudentsȱareȱnowȱregularlyȱadmittedȱ withȱminimumȱpassingȱscoresȱofȱ7.5ȱoutȱofȱ20ȱinȱoneȱsubjectȱonly.32ȱ Inȱthisȱ situation,ȱ UEȱ doesȱ notȱ serveȱ asȱ anȱ effectiveȱ filterȱ eitherȱ onȱ numbersȱ orȱ quality.ȱ Thisȱ hasȱ ledȱ atȱ leastȱ oneȱ leadingȱ Armenianȱ educatorȱ toȱ questionȱ whetherȱanyȱformȱofȱselectionȱbyȱmeritȱisȱneededȱatȱtheȱschool/universityȱ interface!ȱ Theȱ lessonȱ hereȱ isȱ thatȱ theȱ relationshipsȱ betweenȱ supply,ȱ demand,ȱ andȱ qualityȱ inȱ higherȱ educationȱ areȱ complexȱ andȱ soȱ policyȱ makersȱ needȱ toȱ understandȱ fullyȱ theȱ consequencesȱ ofȱ anyȱ decisionsȱ theyȱ makeȱ withȱ regardsȱ toȱ (a)ȱ theȱ natureȱ ofȱ theȱ selectionȱ instrumentsȱ andȱ (b)ȱ theȱ minimumȱcriteriaȱforȱsuccess.ȱ ȱ Conclusionȱ Throughȱ itsȱ sustainedȱ effortsȱ andȱ prudentȱ investment,ȱ Armeniaȱ hasȱ established,ȱ inȱ littleȱ moreȱ thanȱ twoȱ decades,ȱ theȱ foundationsȱ ofȱ aȱ comprehensiveȱ andȱ coherentȱ systemȱ ofȱ studentȱ assessment.ȱ Alongȱ theȱ wayȱ itȱ hasȱ alsoȱ developedȱ theȱ coreȱ ofȱ aȱ potentiallyȱ vibrantȱ assessmentȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 32ȱ Thisȱ isȱ equivalentȱ toȱ aȱ scoreȱ ofȱ justȱ 30ȱ outȱ ofȱ 80ȱ onȱ testsȱ havingȱ significantȱ ‘guessingȱ scores’ȱsinceȱtheyȱareȱcomposedȱmainlyȱofȱfourȬoptionȱmultipleȬchoiceȱitems.ȱInȱGeography,ȱ forȱexample,ȱtheȱ‘blindȱguessingȱscore’ȱfromȱtheȱmultipleȬchoiceȱitemsȱisȱ19/80.ȱ ȱ 40ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ community.ȱ However,ȱ manyȱ challengesȱ remainȱ inȱ maximizingȱ theȱ effectivenessȱ ofȱ theȱ systemȱ andȱ ensuringȱ itȱ bestȱ servesȱ society.ȱ Meetingȱ theseȱ challengesȱ willȱ requireȱ theȱ engagementȱ andȱ actionȱ notȱ onlyȱ ofȱ theȱ ATC,ȱ butȱ alsoȱ policyȱ makersȱ andȱ theȱ broaderȱ communityȱ ofȱ educationalȱ practitioners.ȱ Noneȱ ofȱ theseȱ challengesȱ isȱ uniqueȱ toȱ Armenia.ȱ Armeniaȱ canȱ learnȱ fromȱ theȱ experiencesȱ ofȱ others,ȱ andȱ othersȱ canȱ learnȱ fromȱ theȱ experiencesȱofȱArmenia.ȱ Referencesȱ ȱ Bethell,ȱG.,ȱandȱK.ȱHarutyunyanȱ2008.ȱ “AssessmentȱandȱExaminationsȱinȱ Armenia.”ȱAssessmentȱinȱEducation:ȱPrinciples,ȱPolicyȱ&ȱPracticeȱ15(1):ȱ 107–19.ȱ Bethell,ȱ G.,ȱ andȱ A.ȱ Zabulionis.ȱ 2012.ȱ “Theȱ Evolutionȱ ofȱ HighȬStakesȱ Testingȱ atȱ theȱ SchoolȬUniversityȱ Interfaceȱ inȱ theȱ Formerȱ Republicsȱ ofȱ theȱ USSR.”ȱ Assessmentȱ inȱ Education:ȱ Principles,ȱ Policyȱ &ȱ Practiceȱ 19(1):ȱ7–25.ȱ Black,ȱ P.,ȱ andȱ D.ȱ Wiliam.ȱ 1990.ȱ Insideȱ theȱ Blackȱ Box:ȱ Raisingȱ Standardsȱ throughȱClassroomȱAssessment.ȱLondon:ȱGȱLȱAssessmentȱLtd.ȱ Bloem,ȱ S.ȱ 2013.ȱ “PISAȱ inȱ Lowȱ andȱ Middleȱ Incomeȱ Countries.”ȱ OECDȱ Educationȱ Workingȱ Papers,ȱ No.ȱ 93,ȱ OECDȱ Publishing,ȱ Paris.ȱ Availableȱ at:ȱ http://dx.doi.org/10.1787/5k41tm2gx2vdȬenȱ (accessedȱ 13ȱJulyȱ2015).ȱ Boyle,ȱ W.ȱ 2014.ȱ “PreȬserviceȱ Moduleȱ forȱ Classroomȱ Assessmentȱ andȱ ClassroomȱTesting.”ȱUnpublishedȱmanuscript,ȱFebruaryȱ2014.ȱ ȱ Centerȱ forȱ Educationȱ Projectsȱ (CFEP).ȱ 2009.ȱ “Educationȱ Qualityȱ andȱ RelevanceȱProject,ȱCompletionȱReport.”ȱCFEP,ȱYerevan.ȱ ȱ Clarke,ȱM.ȱ2012.ȱ“WhatȱMattersȱMostȱforȱStudentȱAssessmentȱSystems:ȱAȱ Frameworkȱ Paper.”ȱ SABERȬStudentȱ Assessmentȱ Workingȱ Paperȱ No.ȱ1.ȱWorldȱBank,ȱWashington,ȱDC.ȱ ȱ Governmentȱ ofȱ theȱ Republicȱ ofȱ Armeniaȱ (GORA).ȱ 1999.ȱ “Lawȱ onȱ Education.”ȱGORA,ȱYerevan.ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 41ȱ ———.ȱ2004.ȱ“LawȱonȱHigherȱandȱPostgraduateȱProfessionalȱEducation.”ȱ GORA,ȱYerevan.ȱ ȱ ———.ȱ 2005.ȱ “Decisionȱ #14ȱ Approvingȱ theȱ Studentȱ Assessmentȱ Concept.”ȱGORA,ȱYerevan.ȱ ———.ȱ 2006.ȱ “Decisionȱ onȱ Establishingȱ theȱ Assessmentȱ andȱ Testingȱ Centreȱ andȱ Approvingȱ itsȱ Charter,”ȱ Aprilȱ 14,ȱ 2004ȱ (withȱ subsequentȱamendments).ȱGORA,ȱYerevan.ȱ ———.ȱ2009.ȱ“LawȱonȱGeneralȱEducation.”ȱGORA,ȱYerevanȱ ȱ Hawker,ȱ D.ȱ 2013.ȱ “Designȱ ofȱ aȱ Master’sȱ Programȱ inȱ Educationȱ thatȱ Incorporatesȱ Coursesȱ onȱ Educationalȱ Assessmentȱ andȱ MeasurementȱinȱArmenia.”ȱUnpublishedȱdraft,ȱSeptemberȱ2013.ȱ ȱ Heyneman,ȱ S.ȱ 2009.ȱ “Theȱ Importanceȱ ofȱ Externalȱ Examinationsȱ inȱ Education.”ȱ Inȱ Secondaryȱ Schoolȱ Externalȱ Examinationȱ Systems:ȱReliability,ȱ Robustnessȱ andȱ Resilience,ȱ editedȱ byȱ B.ȱVlaardingerbroekȱandȱN.ȱTaylor.ȱNewȱYork:ȱCambriaȱPress.ȱ InternationalȱAssociationȱ forȱ theȱ Evaluationȱ ofȱ Educationalȱ Achievementȱ (IEA).ȱ 2012.ȱ TIMSSȱ 2011ȱ Internationalȱ Resultsȱ inȱ Mathematics.ȱ Availableȱ at:ȱ http://timssandpirls.bc.edu/timss2011/ȱ (accessedȱ 13ȱ Julyȱ2015).ȱ Ministryȱ ofȱ Educationȱ andȱ Scienceȱ (MOES).ȱ 2008.ȱ Conceptȱ ofȱ Students’ȱ Continuousȱ Assessment:ȱ Generalȱ Statement,ȱ Basicȱ Approaches.ȱ MOES,ȱ Yerevan.ȱ ———.ȱ 2014.ȱ Proceduresȱ forȱ Assessmentȱ ofȱ Studentȱ Progress.ȱ MOES,ȱ Yerevan.ȱ NationalȱCentreȱforȱEducationalȱTechnologiesȱ(NCET).ȱ2015.ȱ“Numberȱofȱ Studentsȱ Studyingȱ inȱ Educationalȱ Institutionsȱ inȱ Armeniaȱ 2011–2015.”ȱ NCET,ȱ Yerevan.ȱ Availableȱ at:ȱ http://stat.armedu.am/?ȱ section=content&id=307&year=2014ȱ(accessedȱ13ȱJulyȱ2015).ȱ ȱ NationalȱInstituteȱforȱEducationȱ(NIE).ȱ2013.ȱ“MethodologicalȱGuidelinesȱ forȱTeachersȱforȱ2013/14:ȱHowȱtoȱApplyȱContinuousȱAssessmentȱinȱ theȱClass.”ȱNIE,ȱYerevan.ȱ ȱ 42ȱ GeorgeȱBethellȱandȱKarineȱHarutyunyanȱ OfficeȱforȱStandardsȱinȱEducation,ȱChildrenȇsȱServicesȱandȱSkillsȱ(Ofsted).ȱ 2008.ȱ “Assessmentȱ forȱ Learning:ȱ Theȱ Impactȱ ofȱ Nationalȱ Strategyȱ Support.”ȱOfsted,ȱLondon.ȱAvailableȱat:ȱhttp://dera.ioe.ac.uk/9309ȱ /1/Assessment%20for%20learning%20Ȭ%20the%20impact%20of%ȱ 20National%20Strategy%20support.pdfȱ(accessedȱ13ȱJulyȱ2015)ȱ ȱ Russiaȱ Educationȱ Aidȱ forȱ Developmentȱ (READ).ȱ 2012.ȱ “READȱ Armenia:ȱ Conceptȱ Noteȱ forȱ Activitiesȱ toȱ Beȱ Carriedȱ Outȱ underȱ READȱ I,ȱ 2012.”ȱREAD,ȱWorldȱBank.ȱ Transparencyȱ Internationalȱ (TI).ȱ 2013.ȱ Corruptionȱ Perceptionsȱ Indexȱ 2013.ȱ Availableȱ at:ȱ http://cpi.transparency.org/cpi2013/ȱ (accessedȱ 13ȱ Julyȱ 2015.)ȱ Worldȱ Bank.ȱ 2011.ȱ “Armenia:ȱ Studentȱ Assessment,ȱ SABERȱ Countryȱ Reportȱ 2011.”ȱ Worldȱ Bank,ȱ Washington,ȱ DC.ȱ Availableȱ at:ȱ http://saber.worldbank.org/index.cfm?indx=2&ctrn=AMȱ (accessedȱ 13ȱJulyȱ2015).ȱ ȱ ȱ ȱ ȱ TheȱDevelopmentȱofȱtheȱStudentȱAssessmentȱSystemȱinȱtheȱRepublicȱofȱArmeniaȱ 43ȱ  ȱ ȱ List of papers published in the current Working Paper series 1. Clarke, M. 2012. “What Matters Most for Student Assessment Systems: A Framework Paper.” (Available in French, Russian, and Portuguese). 2. Ramirez, M. 2012. “Developing the Enabling Context for Student Assessment in Chile.” (Available in Russian). 3. Ramirez, M. 2012. “Disseminating and Using Student Assessment Information in Chile.” (Available in Russian). 4. Liberman, J., and M. Clarke. 2012. “Review of World Bank Support for Student Assessment Activities in Client Countries.” (Available in Russian). 5. Flockton, L. 2012. “The Development of the Student Assessment System in New Zealand.” (Available in Russian). 6. Allen, R. 2012. “Developing the Enabling Context for School-Based Assessment in Queensland, Australia.” (Available in Russian). 7. Castro, M. 2012. “Developing the Enabling Context for Student Assessment in Brazil.” (Available in Russian). 8. Kanjee, A., and S. Acana. 2013. “Developing the Enabling Context for Student Assessment in Uganda.” (Available in Russian). 9. Bolotov, V., G. Kovaleva, M. Pinskaya, and I. Valdman. 2013. “Developing the Enabling Context for Student Assessment in the Russian Federation.” (Available in Russian). 10. Tyumeneva, Y. 2013. “Disseminating and Using Student Assessment Information in Russia.” (Available in Russian). 11. Senghor, O. 2014. “Disseminating and Using Student Assessment Information in The Gambia.” (Available in Russian). 12. Obeidat, O., and Z. Dawani. 2014. “Disseminating and Using Student Assessment Information in Jordan.” (Available in Russian). 13. Bethell, G., and K. Harutyunyan. 2015. “The Development of the Student Assessment System in the Republic of Armenia: Achievements, Challenges, and Lessons Learned.” Forthcoming papers 14. Meckes, L., and J. Gysling. 2015. “Developing and Implementing Learning Standards in Chile.” 15. Al Noaimi, T. 2015. “Developing and Implementing Learning Standards in Jordan.” 16. Flockton, L. 2015. “Developing and Implementing Learning Standards in New Zealand.” 17. Yeap, B-H. 2015. “Curriculum Development and Implementation in Singapore.” In order to maximize effectiveness, every national education system needs a comprehensive and coherent approach to student assessment. The 6$%(5IUDPHZRUNIRUVWXGHQWDVVHVVPHQWLGHQWLÀHVIRXUNH\HOHPHQWV in this regard: classroom assessment, examinations, national large-scale assessment, and international large-scale assessment. Armenia has XQGRXEWHGO\PDGHVLJQLÀFDQWSURJUHVVLQDOORIWKHVHDUHDVVLQFHJDLQLQJ independence from the Soviet Union in 1991. The purpose of this paper is to describe Armenia’s experiences, share the lessons learned along the way, and point out the challenges that countries face in trying to ensure WKDWWKHLUDVVHVVPHQWV\VWHPVUHPDLQÀWIRUSXUSRVHLQDFKDQJLQJ environment. George Bethell, Independent consultant specializing in the policy and practices of educational assessment Karine Harutyunyan, Deputy Minister in the Armenian Ministry of Education and Science The Russia Education Aid for Development Trust Fund is a collaboration between the Russian Federation and the World Bank that supports the improvement of student learning outcomes in low-income countries through the development of robust student assessment systems. Visit the READ website at www.worldbank.org/readtf for additional information.