INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES KNOWLEDGE PACK Learning Management Systems for Education EXPLORE INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES KNOWLEDGE PACK LEARNING MANAGEMENT SYSTEMS FOR EDUCATION INDEX © 2022 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW, Washington, DC 20433 INTRODUCTION Telephone: 202-473-1000; Internet: www.worldbank.org Benefits License: Creative Commons Attribution CC BY 3.0 IGO WHO WHY Use cases This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Structure of solutions Executive Directors, or the governments they represent. This report was also supported with funding from the Global Partnership for Education. Costs considerations The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, Offline or poorly WHAT HOW How to start denominations, and other information shown on any map in this work do not imply any judgment on the connected areas Implementation part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such What has been done boundaries. in other countries Some references may appear in this Knowledge Pack to Logos, Products, Brands or Trademarks belonging to CONCLUSION others not affiliated with the World Bank. They belong to their respective owners/ holders and are used for illustrative purposes only and do not imply any affiliation with or endorsement by them. The World Bank does TO GO FURTHER not endorse, prefer or recommend any of these products. Rights and Permissions ANNEXES Glossary The material in this work is subject to copyright. Because The World Bank encourages dissemination of its References knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. FAQ Please cite the work as follows: EdTech team. 2022. Knowledge Pack : Learning Management Systems for Education. Washington, D.C.: World Bank Group Acknowledgment: Much appreciation goes to Frederico Carvalho who took part in the development of this resource, and to John Moravec, Esther Gacicio and Edmond Gaible worked on an early version Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank of this KP, under the guidance of Iñaki Sánchez Ciarrusta from the EdTech Team. We also want to thank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; Omar Arias and Jaime Saavedra for their overall support, and colleagues Boubakar Lompo, Fadi Daouk e-mail: pubrights@worldbank.org and Shobhana Sosale for providing comments to enrich these resources. Design : Alejandro Scaff, Sarah Kleinmann 2 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Introduction What is a KP? About this KP After reading the main content of this KP, some questions might pop* : Knowledge Packages (KPs) The Learning Management are short, pragmatic guides Systems (LMS) Knowledge on individual topics within pack is meant to be a starting EdTech, meant to provide point in the decision-making sufficient knowledge and process an entity could follow • What are the key considerations between understanding so that non- to decide what is the best open-source and proprietary LMS? technical stakeholders can course of action to choose make key planning, design, and deploy a platform to and procurement decisions support online training. for education. • What are some examples between open-source They can be used as a and proprietary LMS? starting point for the planning of technology deployment to improve education, especially with education ministries. *To know more about these questions and their answers, go to the FAQ available in the annex. 3 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES WHO are the main stakeholders ? RESPONSIBILITY • Assist MOE leadership in the application of KPs are designed Task Team Leaders (TTL’s) & KPs for in-country EdTech programs. Help Bank Project Managers with a human- (non-technical) design Bank-financed projects with practical centered vision. information to include in project documents. This knowledge • Use KP to make key planning, design, and pack is meant to MOE Leadership procurement decisions for in-country EdTech provide sufficient (non-technical) programs. knowledge and understanding so that non-technical • Use KP to make key planning, design, and MOE Program Managers Stakeholders (semi-technical) procurement decisions for in-country EdTech can understand programs. the fundamental elements involved in decisions about Donors, NGOs and Other • Use KP to align with Bank EdTech programs LMS for education. Partners and establish a common EdTech framework. (non-technical) 4 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES HOW Benefits | Use cases WHY is this KP designed ? BENEFITS Deploying a Learning Management System allows to: • Monitor learning progress through summative and formative assessments • Organize and facilitate access to courses in a Digital format to students in any location • Allow students to learn offline and in poorly connected environments. • Provide access to learning content that otherwise wouldn’t be available (international sources) • Support teachers’ delivery of classes in synchronous and asynchronous manner A Learning Management System provides digital learning content to support and • Provide a platform for communication, collaboration engage students that cannot attend between students and teachers class or as a complement to face-to-face classes. • Support the submission and grading of assignments and quizzes • Record information about students’ interaction with content, progress and engagement 5 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES HOW Benefits | Use cases WHY is this KP designed ? USE CASES FULL OFFLINE / CONNECTED SIMULATIONS AND EXAMS / IN-CLASSROOM LOW BANDWIDTH REMOTE LEARNING STEM LABS PROCTORING LEARNING • Packaged content • Bandwidth • Rich practical • In campus • Rich interactive • Mobile client appropriate simulations or remote content • Synch data when content • 3D support examination • Individual and connected • asynchronous • Multiuser • High security user group assignments learning collaboration authentication • Collaboration tools • Collaboration and • Rich interaction • Grade books and project-based and gamification gamification learning • Grade books and gamification Use case scenarios describe what we would like users to be able to do and their context 6 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Structure of solutions | Costs considerations | Offline or poorly connected areas | What has been done in other countries HOW WHAT are the potential solutions? STRUCTURE OF SOLUTIONS KEY FEATURES OF AN LMS A Learning Management System provides digital learning content and data to support and engage students that cannot attend class or as a complement to face-to-face classes. Here are the key features of an LMS: Management of courses and users Management & organization of Content Incentives to drive engagement • Organize & Display Courses • HTML5 – rich web content • Gamification • Enrollment • Video • Grades • Monitor Progress • Documents • Leaderboards • Quizzes • Badges Collaboration & communication Learning Assessment Usage Reporting • Forums • Quizzes • Data collection • Messaging • Assignments • Visualization • Video conferencing (Webinars & • Exams • Analysis Video lessons) • Proctoring • Integration with EMIS 7 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Structure of solutions | Costs considerations | Offline or poorly connected areas | What has been done in other countries HOW WHAT are the potential solutions? STRUCTURE OF SOLUTIONS ELEMENTS OF AN LMS An LMS is at the core of an eLearning system and has to interact with several other systems that can be integrated or standalone. 8 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Structure of solutions | Costs considerations | Offline or poorly connected areas | What has been done in other countries HOW WHAT are the potential solutions? STRUCTURE OF SOLUTIONS FACTORS TO CONSIDER WHEN CHOOSING AN LMS Learning analytics Hosting Offline capability Specific Long-term LMS needs for sustainability the student’s level Integration Open and source VS interoperability Internal commercial Available capabilities to bandwidth manage LMS 9 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Structure of solutions | Costs considerations | Offline or poorly connected areas | What has been done in other countries HOW WHAT are the potential solutions? STRUCTURE OF SOLUTIONS DIFFERENCE BETWEEN OPEN-SOURCE AND PROPRIETARY Open-source – the source code is developed by a community and is made Core LMS Software available for download without any Code versus Cost of Open Source Proprietary cost. It is possible to develop further the license code and create custom functionalities. No support is provided. Moodle License has no Canvas Kolibri Proprietary – the source code is cost Chamilo developed and maintained by a company and its use requires a license which can be free or paid. Blackboard Open LMS (hosted Moodle) Customization of code is done by the Paid license Learndash (WP plug-in) Canvas hosted developer which usually also provides Tribyte support. There are proprietary solutions developed on top of open-source code With open-source code you are responsible for fixing bugs and adding features. (ex. Open LMS, Redhat Linux) With proprietary code you would request support from the supplier. 10 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Structure of solutions | Costs considerations | Offline or poorly connected areas | What has been done in other countries HOW WHAT are the potential solutions? COSTS CONSIDERATIONS BUDGETTING FOR LMS DEPLOYMENT DEPLOYMENT OPTIONS POTENTIAL COSTS TO CONSIDER • Number of concurrent users as a % of total registered • Licensing cost (per year for the amount of registered users or concurrent users) • Hosting costs (IaaS or own platform) • Setup/implementation and consulting fees • Integration with other systems • Data migration • Platform updates and maintenance • Technical Support • Change management • Skills training 11 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Structure of solutions | Costs considerations | Offline or poorly connected areas | What has been done in other countries HOW WHAT are the potential solutions? COSTS CONSIDERATIONS TYPICAL PRICING MODELS PER LEARNER, PER PER LEARNER, PER USE PER COURSE LICENSING FEE MONTH The most common LMS solutions can include Some LMS vendors have Companies can purchase payment option is a more features, lessons, a pay per course option a license to install LMS per learner, per month modules and tools than for employees that need software on-premise. subscription fee. This needed. A per learner, special certification for The licensing fee can figure is either calculated per use pricing model their industry or job be paid annually or up- based on the total number allows clients to opt out function. Vendors might front as a one-time fee. of users in the system or of certain parts of the have content available There are often additional the number of active users software and pay only for in an internal library or implementation expenses enrolled in courses. Fees what they use. partner with third-parties for installation, data are generally $5 or less This could mean being to deliver lessons through migration and training, per learner per month. charged based on the their LMS platform. and there may also be modules in use, the Pricing is very variable. a charge per learner. number of active accounts Licensing fees generally in use or only the content range from $500 to that’s being used by $20,000, and setup can those accounts. Prices cost over $25,000. range from $1 to $10 per learner, per use. Four standard LMS pricing methods 12 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Structure of solutions | Costs considerations | Offline or poorly connected areas | What has been done in other countries HOW WHAT are the potential solutions? LMS FOR OFFLINE OR POORLY CONNECTED ENVIRONMENTS DEVICE BASED LMS CLIENT When connectivity is not available or has poor quality, access to learning content and courses can be provided offline through either a local server or a LMS client installed in the user device which caches the content locally. One of the most common options when connectivity is not Example of Kolibri available is to install the learning content directly into the learning device, especially when students need to use it at home. This can be done by simply copying files into a structure of folder and by using an application that allows the content to be organized in a user interface. Although this is a “better than nothing” option and something cheaper than printing and distributing books (especially if the devices already exist), it has proven to have severe limitations namely: 1. Difficult to update or add new content 2. No feedback provided on how students interact with the content 3. No interaction between the students, teachers and peers 13 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Structure of solutions | Costs considerations | Offline or poorly connected areas | What has been done in other countries HOW WHAT are the potential solutions? LMS FOR OFFLINE OR POORLY CONNECTED ENVIRONMENTS LOCAL LMS SERVER When connectivity is not available or has poor quality, access to learning content and courses can be provided offline through either a local server or a LMS clients installed in the user device which caches the content locally. More and more projects are using “school Servers” to host Example of C3 Cloud learning content and shared applications. These devices are installed in the school and provide access to learning content through a local instance of the LMS and a wifi connection provided by the device itself or a separate Wifi Access Point. The server can be connected to internet through a mobile network or taken periodically to a connected location to get updates of content, applications but also upload student’s usage analytical data, providing important feedback to content managers. This solution can also be a source of content for devices tat have a local LMS client allowing the update and replacement of content by simply having the student connecting to the school network. 14 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Structure of solutions | Costs considerations | Offline or poorly connected areas | What has been done in other countries HOW WHAT are the potential solutions? WHAT HAS BEEN DONE IN OTHER COUNTRIES EXAMPLES OF WORLD BANK FUNDED LMS IMPLEMENTATIONS Tunisia Digital Turkey: Safe Schooling and Vietnam Enhancing Teacher Transformation for User- Distance Education Project.  Education Program Centric Public Services 15 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES How to start | Implementation HOW to implement next steps ? HOW TO START ESSENTIAL FEATURES TO TAKE INTO ACCOUNT FROM THE BEGINNING Choose an LMS that is modular and able to seamlessly integrate with other solutions and share data in standardized format. Look for solutions compliant with one of the standards below. There are 2 main entities defining data interoperability standards: Advanced Distributed Learning IMS Global Learning Consortium Initiative • SCORM (Sharable Content Object • Caliper Analytics® Specification Reference Model) and Sensor API™ programming • xAPI (Experience Application interface Programming Interface) Around each of these specifications there is an ecosystem of applications and related specifications. Caliper is being developed by a closed consortium, and SCORM and xAPI in an open process. 16 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES How to start | Implementation HOW to implement next steps ? HOW TO START WHERE TO START Analise what Identify the solutions, Estimate problems you existing effort to Evaluate Identify what want to solve and future Perform a transition to a possible exists today and how content and TCO analysis new solution solutions (Use Case expertise from existing Scenarios) is available legacy locally 17 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES How to start | Implementation HOW to implement next steps ? IMPLEMENTATION KEY DECISION AND IMPLEMENTATION STAKEHOLDERS POLITICAL TECHNICAL PEDAGOGICAL • MoE Top management • Infrastructure management • Teachers • MoE Finance department team • Pedagogy team • MoE Procurement • LMS management team • Curriculum development team • HE institutions management • Digital content development • Content development team • EMIS administrators team • EMIS administrators 18 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES How to start | Implementation HOW to implement next steps ? IMPLEMENTATION USE PROVEN APPROACHES – EXAMPLE OF PROFUTURO Test before committing to a particular solution using MVP approach. 19 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES HOW Conclusion WHAT WHO WHY Establish the team responsible for Describe in detail the type of users and defining and implementing the LMS learning experiences you want to support Choose your platform and Establish your TCO budget and project WHAT implementation model. plan HOW 20 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES To go further CLOUD OF KPs R E L AT E D S O U R C E S AI/ML Devices Cloud Adaptive Learning Connectivity For further resources refer to the Digital Infrastructure Digital identity BE DATA Annexes Data visualization DRIVEN ENGAGE the NRENs ECOSYSTEM Procurement Ecosystem Startups Data collection EMIS AssistiveTechnologies LMS LEARNER DESIGN and Mobile based S T AY C O N N E C T E D Computer based ACT AT SCALE, Digital ASK WHY? FOR ALL Assessment Literacies Follow us on Twitter Digital Content EMPOWERED TEACHERS Subscribe to our podcast channel Teachers Competencies Spotify & Anchor More updates on Medium Subscribe to our EduTech Newsletter EdTech website OTHER EXISTING RELATED KPs EMIS 2.0 Digital content Procurement Ecosystems 21 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Glossary | References | FAQ HOW Annexes WHAT GLOSSARY Adaptive learning is a segmentation Caliper Analytics® is a standard from Google) that operates and maintains of content and skills in modules so the IMS Global Consortium® which the infrastructure to ensure that the student himself or in cooperation enables institutions to collect learning it’s operating and online. The client with the teacher and led by learning data from digital resources. Adopters maintains the software and data analytics, can choose what is the most of the standard can use Caliper to related to the LMS. productive to do next (progress or better understand and visualize cover a learning gap, etc) learning activity and product usage Commercial software requires data (1) . the purchase of software licenses Assessment systems may be divided and generally does not afford users into two groups as: Certification and compliance privileges to analyse source code, make Standalone systems without LMS, and management – Some industries modifications, or share the software. Integrated systems with the LMS. or specific jobs require employee certification or compliance training. Content development – Organizations Asynchronous learning lets students This feature manages and tracks can develop personalized course access course materials at different whether employees have completed all content, either using templates or times and locations. The main requirements and can generate reports from scratch. Course authoring tools benefit is that users can learn at their for auditing. let administrators design courses own convenience. Because users with multimedia, such as videos, aren’t bound to a class schedule, Cloud hosting - The LMS is hosted PDFs, infographics, animations and organizations often set deadlines to on an infrastructure operated by a PowerPoint presentations. motivate users to complete courses. third party (e.g., Amazon, Microsoft or 22 Source: https://www.betterbuys.com/lms/lms-pricing-guide/ | https://ed.link/community/caliper-analytics/ INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Glossary | References | FAQ HOW Annexes WHAT GLOSSARY Content library – Many LMS solutions Learning analytics (LA) is a umbrella specific courses. include an out-of-the-box content concept, defined by George Siemens library with premade learning (2012) and can be the tool for teachers Learning Management System materials. The library may cover to have an overview over learning (LMS) is a software application for a variety of topics, such as safety successes and difficulties for a whole the administration, documentation, regulations, customer service and class of students, instead of having that tracking, reporting, automation and management techniques. overview for only a couple of them (if delivery of educational courses, even that). training programs, or learning and Gamification – Also known as game- development programs. LMS also based learning, gamification provides Learning Environments (LE) refer to provides the Course and Content incentives for students to stay engaged the diverse physical or virtual locations, Management System component and by offering points, badges and other contexts, and cultures in which others. achievements. Students are motivated students learn. by competition, entertainment and the Mobile learning gives students access casual nature of games. Learning management helps users to LMS materials through a mobile app organize and automate education and or web browser. This feature allows H5P makes it easy to create, share training. It includes tools for managing remote and asynchronous learners to and reuse HTML5 rich and interactive user info, scheduling, alerts, messages complete courses on the go. web content and applications that is and delivering content. Administrators responsive and mobile friendly. can also set prerequisites for training and allow users to self-enroll into 23 Source: https://www.researchgate.net/post/What_is_Learning_Analytics_latest_research_trend INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Glossary | References | FAQ HOW Annexes WHAT GLOSSARY Open-source software is generally SCORM (Sharable Content Object Testing and assessments – Tests, available to anybody to use freely (or Reference Model) was released in assessments and quizzes see whether at low cost), analyze source code, make 2001 and is the most widely used individual students have retained modifications, and share under terms standard in remote learning. SCORM- what they learned. They can be given adopted by the software creator. compatible content objects can be after each lesson or at the end of the assembled into units of study that run course. Analytical tools can identify Performance tracking and reporting in a browser-based LMS. proficiencies and learning gaps. functions help determine if the overall learning program is effective. Self-hosting - The client owns and xAPI (Experience Application Organizations can analyse how users operates the infrastructure that Programming Interface) was released have performed in their courses and host the LMS on premise or hosted in 2010 as a significant update to how much time they spend learning. externally. SCORM, one focused on improving information about a student’s SaaS (or Software as a Service) -The Synchronous learning requires achievement. xAPI supports recording supplier hosts the LMS, providing students to learn at specific times, information about a student’s progress services such as customization prior to either online or in a physical classroom, through an online course and also launch and trouble-shooting after the which can foster collaboration student learning that takes place LMS is in operation. The client (MOE, between employees. However, it can without a course and even without a university, school, etc.) licenses the be difficult to coordinate and schedule computer. software, typically paying an annual fee synchronous learning. per user. 24 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Glossary | References | FAQ HOW Annexes WHAT REFERENCES • IMS Global - https://www.imsglobal.org • Advanced Distributed Learning (ADL) - https://adlnet.gov/ • https://elearningindustry.com/elearning-standards-scorm-aicc-xapi-cmi5-ims-cartridge EXTRA RESOURCES • https://ed.link/community/caliper-analytics/ • Learning Analytics Community Exchange Comparing xAPI and Caliper Learning Analytics Review 7 ISSN: 2057By: Dai Griffiths and Tore Hoel - https://lace.apps.slate.uib.no/wp-content/uploads/2016/09/LACE- review07_xapi-caliper_v1-1.pdf • How to include youth in low bandwidth areas - https://www.s4ye.org/s4ye-publications • TCO - https://www.webanywhere.com/2018/06/23/calculating-the-total-cost-of-ownership-for-totara- vs-a-proprietary-lms/ | https://medium.com/@Software3E/open-source-vs-proprietary-lms-a-bird-view- c905c3218e5d 25 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Glossary | References | FAQ HOW Annexes WHAT FAQ • What are the key considerations between open-source and proprietary LMS? • What are some examples between open-source and proprietary LMS? 26 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Glossary | References | FAQ HOW Annexes WHAT FAQ Back to What are the key considerations between open-source and proprietary LMS? Category Open Source LMS Proprietary LMS (Free or Paid) Licensing Free / Small cost for Software licensing. Free or Annual licensing fee per user enrolment. Additional features added through plugins. These can be free, Additional payment would be asked by vendors to initiate the Features paid or require custom development. development as per your requirements Integrations with leading solutions are usually available but Integrations Some LMS offer integration through custom development may be offered at an extra cost. Hosting Usually hosted on infrastructure on prem or IaaS Hosted on own server or vendor server (SaaS) Support is usually provided by the community or paid service Tech-support is usually offered by the vendor or its official Support providers. partners. Scalability Increasing users doesn’t increase LMS licensing costs Increasing users can increase licensing costs. Security fixes relies on open-source community and frequency of Security Security checks and updates are provided by the vendor. own maintenance and updates Free core LMS software but requires own hosting and technical Usually offered as a SaaS, involves a set-up cost and a perpetual Keys Trade- skills to customize, enrich features and deploy. Sometimes subscription license (per user basis) but includes hosting, offs support is limited to few knowledgeable people in the world. updates, support and customization services. Can also be Some companies offer hosting and other services for a fee. offered over an open-source platform. 27 Source: 3E software Solutions Blog 1 and Blog 2 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Glossary | References | FAQ HOW Annexes WHAT FAQ Back to What are some examples between open-source and proprietary LMS? Examples of Open-Source LMS’s Examples of Proprietary LMS’s • Moodle™ : Moodle is an open source LMS free to • Blackboard LMS : Blackboard is a leading commercial download. Moodle is also offered as a subscription proprietary solution. Fully compliant with IMS guidelines cloud solution and has several add ons such as Moodle it provides many functionalities such as comprehensive App allowing courses to be done offline. analytics and engagement tools , assessments and certifications and collaboration tools, etc. • Chamilo : Open LMS offers a highly curated, enhanced, SaaS-based versions of Moodle™ and full • Learndash : Learndash is a Wordpress LMS plug in that implementation and user support services. turns a WordPress site into a learning management system. It supports SCORM and xAPI and many other • Canvas : Canvas is an open-source learning functionalities through third party plug-ins. management solution created by Instructure. Instructure offers a reliable hosted and supported • Open LMS : Open LMS offers a highly curated, version for a set-up and licencing cost. enhanced, SaaS-based versions of Moodle™ and full implementation and user support services. Is it important to note that Open Source doesn’t always mean “free” software as some solutions based on open- • Kolibri : Kolibri is a free proprietary offline-first learning source platforms are offered as a paid service. platform that runs on a variety of low-cost and legacy devices. Learning Equality offers customization and training for a fee. Google Classrooms : Although it contains certain features of an LMS, Google Classroom is considered a Classroom Management system and not a Learning Management System. 28 INTRODUCTION WHO WHY WHAT HOW CONCLUSION ANNEXES Supported with funding from