1.　Introduction

The COVID-19 pandemic has significantly impacted education worldwide, forcing higher educational institutions to shift from traditional in-person learning to remote and online learning [1]–[3]. In this context, the Learning Management System (LMS) has emerged as the most logical and practical solution for institutions to continue providing education during the pandemic. LMS is a feature-rich educational platform that allows faculty and instructors to manage courses and also allows students to access course materials, submit assignments, and communicate with their instructors and peers. The Institute for Information Management and Communication (IIMC), Kyoto University has provided an LMS platform called PandA (People and Academe), which is based on Sakai LMS [4], an open-source LMS platform developed by the Apereo Foundation.

Before the pandemic, the LMS was only occasionally used by some courses, and its usage was limited at Kyoto University. Most courses tend to take attendance in person by posing paper quizzes, and course materials and assignments were handed out during the class. Therefore, the existence and usability of the LMS were not well recognized by both faculty and students. Things have changed since the COVID-19 started spreading. Like other universities worldwide, Kyoto University switched from face-to-face to online courses in April 2020. Class attendance was taken using assignment and test&quiz features on the LMS, and students were required to submit them before the deadline. At the university, instructors have the option to create course sites at the LMS for their courses, and the number of courses using the LMS has increased significantly since the pandemic began. In the fiscal year 2020, a total of 14,841 undergraduate and graduate courses were provided [5], [6]. Among these, 9,852 classes were integrated with the LMS, constituting 66.4% of all courses. In contrast, in the pre-pandemic fiscal year 2019, out of 14,817 courses, only 1,740 were connected to the LMS, making up merely 11.7% of the total courses.

Despite the widespread adoption of the LMS in online courses, there were some notable usability issues with the LMS for students. A primary issue at Kyoto University was the lack of a feature in the LMS that allowed students to view all available assignments across their enrolled courses. This shortcoming led to substandard user experiences, particularly for students enrolled in multiple courses. In 2020, a survey conducted by the Center for the Promotion of Excellence in Higher Education, Kyoto University, revealed that over 40% of teachers asked students to submit assignments using the LMS [7]. Consequently, students were required to check the LMS daily to see if there were any assignments or test&quizzes available for each course, which was tedious and time-consuming. It was, therefore, necessary to have a feature that enables students to view assignments from all courses in one place.

Table 1 provides a feature comparison across various LMS platforms used in Japan such as Sakai, Moodle, WebClass, manaba, CoursePower, and Blackboard [4], [8], [9], [10], [11], [12]. The table highlights the absence of features that are crucial for enhancing student usability and engagement. For instance, features such as a list of assignments and notifications for new assignments are not available in most LMS platforms, including Sakai. This lack of functionality underscores the necessity of developing tools that enhance the usability of the LMS.

In this paper, we present Comfortable PandA, a web browser extension that resolves the issues mentioned above by enhancing the usability of the Sakai LMS from a student's perspective. The features implemented and the experience gained from developing this extension are expected to provide valuable insights for improving the usability of other LMS systems as well.

The rest of this paper is organized as follows. In Section 2, we describe the extension's development, the features of the extension, and the development process. In Section 3, we describe the publication of the extension and the number of students using the extension. In Section 4, we introduce the official recognition of the extension by Kyoto University and outline regulations for developing enhancements to the LMS. In Section 5, we evaluate the usability by a survey targeting students who have installed the extension. Finally, Section 6 concludes this paper.

2.　Development of the LMS Extension

2.1　Overview of the Developed Extension

The LMS at Kyoto University provides a wide range of tools for faculty and students, such as course material, assignments and test&quizzes, grade books, and LTI (Learning Tools Interoperability) tools. These tools aim to provide an educational platform and enhance the learning experience for both faculty and students. Originally, the design of the LMS was primarily focused on meeting the needs of faculty, such as distributing course materials, collecting assignments, and managing grade books. As such, faculty members have been the primary users of the LMS at Kyoto University. However, this approach has overlooked the convenience of student users, as their feedback has not been sufficiently incorporated into the design process. Additionally, resources for enhancing the LMS, incorporating both faculty and student perspectives, have been limited at Kyoto University.

One such example of the lack of student-centric design is the absence of a feature that allows students to view all available assignments and test&quizzes of all registered courses in one place. The assignment and test&quiz tools provided by the LMS are designed to be used within each course page. Each course has its own assignment and test&quiz tools, and students are able to check assignments and test&quizzes only specific to that course. If students are taking multiple courses, they have to check each course page to see if any assignments or test&quizzes are available. This has resulted in students often missing assignments or test&quizzes, which can be a severe problem for some students.

To address the issue mentioned above, the first author, a student at Kyoto University, has developed Comfortable PandA, a web browser extension that enhances the usability of the LMS. The naming of the extension Comfortable PandA reflects the hope that it will make the LMS PandA more comfortable to use for students. The web browser extension is a software module that extends the functionality of a web browser. The main advantage of using a web browser extension is that it directly integrates the functionality mentioned above onto the browser page, such as adding and modifying the page's content. Therefore, the web browser extension does not require modifications to the LMS itself, and it can be installed and used by students without the need for any special permissions from the authentication server of the university. In developing the extension, the first author has decided to identify and compile a list of the key functionality students seek in a tool that enhances the usability of the LMS.

• (1)Easy to install and use
• (2)List all assignments and test&quizzes in one place
• (3)Visualize the due dates of assignments and test&quizzes
• (4)Notify students of new assignments and test&quizzes

Comfortable PandA extension was developed to realize the functionalities listed above using the REST API provided by the LMS.

2.2　REST API of the LMS

The Sakai LMS, which serves as the basis for the LMS at Kyoto University, provides a REST(Representational State Transfer) API (Application Programming Interface) for accessing information such as assignments and test&quizzes. This REST API was initially designed to be used to populate the LMS with information from external sources, such as the university's student information system [13].

The developed extension uses the REST API to realize the functionality listed above and enhances the LMS's usability. Whenever the LMS page is accessed, the extension requests the REST API to retrieve information such as assignments and test&quizzes. This REST API requires authorization to access such information and is only accessible to users who have logged in to the LMS. However, by calling the REST API from the extension, the browser automatically performs the authorization process using the authorized session information stored in the browser. After the authorization process, the extension sends requests and retrieves information from the REST API. The retrieved information is then parsed and displayed on the LMS page by extending the page's DOM (Document Object Model). Fig. 1 shows the relationship between the REST API and the extension.

2.3　Features of the Developed Extension

Students who have installed the extension can access various features that enhance the usability of the LMS. In this section, the key features of the extension are described in detail.

2.3.1　List of Assignments and test&quizzes

The most important feature of the extension is miniPandA, a list of assignments and test&quizzes of all registered courses, as shown in Fig. 2. A newly added menu button, shown in Fig. 3, is placed on the top bar of the LMS page, allowing students to open the miniPandA list. The list contains all assignments, test&quizzes, and memos. Items related to assignments and test&quizzes are obtained from the REST API, whereas memos are manually added by users. Each item displays the course name, due date, and detailed assignment or test&quiz description. These items are organized by their due dates, prioritizing the most urgent tasks displayed at the top of the list. Furthermore, items are differentiated by a four-color scheme―red, yellow, green, and gray―to represent their due status visually. Color schemes are assigned as follows:

• ・RED: Due dates within a day
• ・YELLOW: Due dates within five days
• ・GREEN: Due dates within 14 days
• ・GRAY: Due dates beyond 14 days

This color scheme is designed to help students prioritize their tasks by highlighting the most close-due assignments and test&quizzes. Additionally, each item incorporates a checkbox, enabling students to track the submission status of their assignments and test&quizzes. Students can manually mark these checkboxes upon completion or be automatically checked upon submission. Items marked as completed are sorted into a separate group, facilitating a clear distinction between completed and uncompleted tasks.

Moreover, the list can be accessed outside the LMS page by clicking the extension pop-up icon on the browser toolbar, as shown in Fig. 4. The pop-up presents a similar list of assignments and test&quizzes based on information previously retrieved from the REST API, allowing students to check their tasks without accessing the LMS page.

2.3.2　Intuitive Coloring of Course Tabs

This feature colors the tabs of the registered course pages displayed on the LMS's top bar, using the same color scheme described in the previous section. The default course tab without the extension is not colored, as shown in Fig. 5 (a). When a user installs the extension, the course tabs are colored by the due dates of assignments and test&quizzes, as shown in Fig. 5 (b). In cases where a course has multiple assignments or test&quizzes, the tab color is determined by the item with the closest due date. Importantly, completed assignments and test&quizzes are excluded when determining the color of the course tab.

2.3.3　Notification Badge

The extension also provides a feature that notifies students of new assignments without the need to access each course page. As demonstrated in Fig. 5 (b), specific course tabs are marked with a red notification dot in the upper-left corner, indicating the release of new assignments. The extension calculates the difference between the assignments of the previous and current accesses to the course page. Based on this calculated difference, the red notification dot is displayed when new assignments are released. The red notification dot is removed once the student accesses the course page with the notification dot. This feature enhances the students' awareness of new assignments, minimizing the likelihood of missing assignments.

2.3.4　Caching of REST API Responses

Inside the extension, multiple requests are sent to the REST API to retrieve information on assignments and test&quizzes. As these requests are made for each course, many requests may be sent simultaneously (e.g., 30 requests for 15 courses per access). Concerns were raised that frequent browsing of the LMS page by students using the extension could significantly increase traffic load on the LMS. To mitigate this issue, the extension implements a caching feature that stores the REST API responses to reduce the number of requests sent to the LMS.

Before introducing this feature, the extension sent a request every time the LMS page was accessed or refreshed. With the caching feature, the extension evaluates the time elapsed since the last request. If this duration is shorter than a predefined interval, the extension retrieves the information from the cache instead of sending a new request to the REST API.

The minimum interval for applying the cache is shown in Table 2. These two intervals are desiged in consideration for their respective usage scenarios. Typically, the assignment feature is frequently utilized for tasks with short deadlines, such as pop quizzes for attendance, hence the cache interval is set to two minutes. On the other hand, the test&quiz feature is intended for tasks with longer deadlines, such as final exams or term papers, thereby the cache interval is set to ten minutes. Although these intervals are set as default, users can modify them from the extension settings page. By implementing this feature, the number of requests sent to the LMS is considered to have decreased, thereby reducing the load on the LMS.

2.3.5　Universal Color Scheme

Users can freely customize the color scheme mentioned in Section 2.3.1. Fig. 6 shows a part of these customization options. This enhancement was implemented in response to user feedback. Several students expressed challenges in distinguishing each color, while others requested the color scheme to be customizable. Considering these user requests, the extension allows users to customize the color scheme according to their preferences.

2.3.6　Other Features

In addition to the features described above, the extension incorporates user feedback to implement necessary features, such as switching to a dark theme and changing the number of course tabs displayed on the LMS's top bar. This feedback is collected directly via email or by regularly checking social media posts and is used as a reference for further improvement of the extension.

2.4　Open Source Development

The developed extension is being developed as open-source software (OSS). Adopting an open-source approach ensures transparency in the source code itself and enables other students to freely and independently implement new functionality on top of the existing codebase. The codebase of the developed extension is available as a public repository on GitHub [14] and is licensed under the Apache License 2.0.

GitHub provides many features that facilitate open-source development, such as Issues and Pull Requests features [15]. By utilizing these features, students can request new functionalities, report bugs, and propose improvements to the extension. For instance, a student suggested a feature for automatically marking assignments as complete upon submission detection. This suggestion was submitted as an Issue, and the corresponding code modifications were contributed through a Pull Request by the student. The code changes were reviewed by the developers, including the first author, and after confirming that there were no issues, the changes were merged into the codebase. This development cycle of open-source software allows the extension to be continuously improved by not only the developers but also other students.

3.　Publication of the Extension

3.1　The Initial Release

Initially, the developed extension was not intended for public release; it was solely developed for the first author's personal use. However, during its development, the first author received interest and feedback from other students who were also struggling with the usability of the LMS. Consequently, the first author decided to release the extension to the public with the hope that it would be helpful to other students. The extension's reputation and ease of use were spread through social media, and some teachers introduced the extension in classes. Despite only having minimal functionality at the time of initial release, the extension was used by approximately 100 students within a few weeks of its release. Given the positive feedback from students, the first author decided to continue developing the extension and to support a broader range of browsers.

3.2　Support for Major Browsers

The initial release of the extension was only compatible with Google Chrome, as it was thought to be the most used browser among students. In addition to Google Chrome, the extension now supports four major browsers: Mozilla Firefox, Microsoft Edge, and Apple Safari. Most of the extension framework is universally reusable in the current major browsers, making it possible to operate on browsers other than Google Chrome with only minor modifications to the extension's original code. Although a few differences exist among browsers' rendering engines, slight adjustments, such as changes in the CSS(Cascading Style Sheets), are required to ensure the extension is displayed correctly on each browser. However, these changes to bridge browser differences are made to be filled automatically during the extension release phase, making it possible to support multiple browsers with minimal development effort.

3.3　Number of Users Using the Extension

Fig. 7 illustrates the trend in the number of users using the extension. The figure presented represent the total daily active users as recorded by the extension platforms of various browsers: Google Chrome, Mozilla Firefox, Microsoft Edge, and Apple Safari. These numbers reflect unique user accounts that have accessed the extension at least once within a specific day, as reported by the respective browser platforms. It is important to note that the each browser platform does not distinguish between users who access the extension via multiple browsers. Consequently, the active user count depicted in Fig. 7 indicates the total number of unique users per browser platform, potentially counting the same individuals multiple times if they use the extension on more than one browser.

From Fig. 7, it can be observed that the number of active users has been continuously increasing since its release in May 2020, and the number of users exceeds 9,000 as of March 2023. This number is approximately 75% of the total undergraduate students at Kyoto University. The active user count decreases during long vacation periods in August, September, March, and April due to the reduced opportunities for users to access the LMS. After these long holidays, the number of users tends to recover, and an increase in new users is typically observed as a new semester begins. The number of users using the extension is expected to continue to increase as the extension becomes more widely known among students.

Fig. 8 presents the distribution of users across different browsers. The browser with the most users is Google Chrome, accounting for 60% of the total. This is followed by Safari, which comprises 30% of the total user base. The remaining 10% is split between Microsoft Edge and Firefox, with Microsoft Edge accounting for 9% and Firefox accounting for 1%. Notably, within the Safari user base, 74% utilize the extension on mobile devices such as iPhones and iPads. Given that the Safari version of the extension is currently the only one available on both mobile devices and PC browsers, this high proportion of mobile users underscores the demand for enhancements to the LMS not only on PC browsers but also on mobile platforms.

3.4　Collaborative Development of the Extension

Initially, the extension was developed by the first author alone, a student at Kyoto University. As the extension gained popularity among students, the first author began receiving requests for new features and improvements. In response, the first author decided to co-develop the extension with other students, forming a development team to enhance the extension based on user feedback and requests. Within a month of its initial release, the first author invited peers from the same classroom who were also interested in improving the LMS to join the development team. The team shared issues and tasks on GitHub and collaborated on implementing the new features and improvements. The team now includes five students, from undergraduates to graduates, who actively contribute to the development. An important aspect of collaborative development is encouraging students who request new features to actually write code and participate in the development process. This approach not only allows students to contribute to the development but also ensures that the community can sustain the extension's development even after the members graduate.

4.　Collaboration with Kyoto University

To ensure that students can confidently use the developed extension, the first author decided to obtain official recognition of the extension from the university. When the extension was initially released, due to a comparatively small number of users, collaboration with the university was not considered. However, as reputations about the extension spread among students, questions arose from students considering using the extension, such as “Is this extension safe to install?” and “Is it okay to use the LMS with this extension installed?”. Given these concerns, obtaining official recognition from the university became necessary to ensure students could confidently use the extension and to encourage more students to use the extension.

4.1　Code for PandA ― Regulations for Developing Enhancements to the LMS

In recognition of the extension by the university, IIMC, Kyoto University established twelve requirements of regulations on May 2021, called Code for PandA [16] to guide user-driven software development to improve the LMS [17]. As of April 2024, two cases are authorized including Comfortable PandA. The authorization is required to renew every fiscal year. Although the twelve requirements were uniquly determined by internal discussions at IIMC, the intention is similar with Code for Sabae [18], Code for Japan [19] and other civitech activities. Code for PandA encompasses two meanings: “code” as in software designed to enhance the LMS, and “code” as in the rules governing such development activities. Establishing these rules facilitates the development process for those creating software to enhance the LMS and enables the university to certify software developed under these rules. The certification requirements for Code for PandA are shown in Table 3. These requirements include the public disclosure of the developer's name, software development as an OSS project, and the requirement to establish privacy policies.

With the establishment of Code for PandA, the university can officially recognize software that meets these requirements as a case study of the LMS enhancement. Consequently, the developed extension is adjusted to fulfill each requirement of Code for PandA and was officially recognized as the first case of official recognition in May 2021. The official recognition from the university provided an opportunity for a broader audience, including faculty members, to become aware of the extension, leading to increased usage of the extension by students.

4.2　Comfortable Sakai: Presented Beyond Kyoto University

Upon receiving official approval for the extension, the first author developed a new extension called Comfortable Sakai. This new browser extension is designed for broader use, extending beyond Kyoto University to support various Sakai schools worldwide. Since Comfortable PandA extension was developed specifically to work at Kyoto University, it was incompatible with other instances at Sakai schools. Moreover, the extension was not internationalized, making it difficult for students at universities outside of Japan to use the extension. Therefore, the Comfortable Sakai extension was developed to be compatible with any Sakai LMS and to support multiple languages. The essential features of Comfortable Sakai extension are the same as those of Comfortable PandA extension. However, the support for i18n (internationalization) and l10n (localization) was added, and the dependent feature for Kyoto University was removed. This new extension is also developed as an OSS and is available at GitHub [20] for anyone to install and modify the source code.

Comfortable Sakai extension has been officially introduced as a case study by other universities [21]. For example, at Nagoya University in Japan, the extension was introduced as a tool to enhance the usability of the LMS (Fig. 9) [22].

4.3　Other Dissemination Activities

Given the reputation of the extension outside of Kyoto University, we had the opportunity to give an invited talk at various conferences and seminars. In Educational Institution DX Symposium organized by NII, we introduced the development of the extension enhancing the usability of the LMS from the student's viewpoint and highlighted the significance of the collaboration between the university and its student [23]. In Open Apereo conference, an international conference on open-source software for education, we suggested that Comfortable Sakai could be co-developed with other universities [24] and that the extension could be used as a platform of students' participation in the enhancement of the LMS [24]. We hope that both Comfortable PandA and Comfortable Sakai extensions will act as a catalyst for the development of comparable extensions or tools aimed at enhancing the usability of the LMS by incorporating the students' perspective.

5.　Evaluation of the Developed Extension

5.1　Evaluation Method

A survey was conducted targeting students who use Comfortable PandA extension to investigate the usage status and user satisfaction of each feature available in the extension. The survey was implemented using Google Forms, and the responses were collected anonymously. Announcements for survey participation were made on the top page of the extension between April and May 2023. There were 431 responses in total, with 89.3% being from undergraduate students, while the remaining 10.6% were from graduate students at Kyoto University. It is important to note that the findings may be biased, as the survey was conducted among students who had already installed the extension. Therefore, when interpreting the results, it is essential to consider that the survey results may reflect the opinions of students who are more likely to use the extension frequently.

5.2　Survey Results on Usage of the Extension

Table 4 presents questions related to the usage of the extension. These questions aimed to gather insights on the methods through which students discovered the extension, how frequently they utilized it, and their primary motivations for installing it. For question 1, the answers were selected from “Via social media such as Twitter”, “By a friend”, “Introduced in class”, “Introduction page of the extension”, and “Others.” The most common response, accounting for 49.4% of the total, was discovering the extension via “Social media such as Twitter”. Coupled with “Introduced by a friend” at 42.9%, it becomes clear that a majority of users, over 90% of respondents started using the extension after hearing about it from other students. It can be seen that introducing and promoting the extension via social media was effective, especially during the COVID-19 pandemic, where in-person interactions between students were limited.

For question 2, the answers were selected from “Every day or almost every day”, “2–3 times per week”, “About once per week”, “About once per month”, and “Hardly used”. Among the responses, 85.1% of students answered they used it “Every day or almost every day”, 12.3% answered “2–3 times per week”, and 1.5% answered “About once per week”. From this, it can be inferred that over 97% of students utilized the extension at least a few times a week.

In question 3, respondents were asked to select multiple answers regarding the features they primarily use in the extension. Fig. 10 illustrates the usage status for each feature. Since this question allowed multiple selections, the sum of each item does not add up to 100%. Each percentage represents the proportion of respondents who answered that they use the corresponding feature. The most primarily utilized feature is the “List of assignments and test&quizzes”, which was used by 91.3% of respondents. This is followed by the “Coloring of course tabs” feature, used by 53.0% of respondents, and the “Notification of new assignments and test&quizzes” feature, used by 23.4% of respondents. These three features were initially developed with the intention of improving the LMS's usability, and it is encouraging to see that they are being utilized by many students. It is also remarkable to note that the “Submission detection” feature, which was developed by volunteer students, was used by 26.0% of respondents. This high usage rate highlights the importance and effectiveness of the open-source development approach, which allows anyone to contribute to enhancing the usability of the LMS.

Based on the insights derived from these survey questions, it is evident that the extension's popularity was primarily propagated through social media and word of mouth. Furthermore, the data shows that the majority of students used the extension frequently to use the enhanced features. These suggest that the active promotion of the extension, along with its high usability, played key roles in its broad adoption. In the next section, survey results concerning the features available in the extension are evaluated in detail.

5.3　Survey Results on the Extension's Features

This section presents the survey results regarding the features available in the developed extension. We attempted to quantitatively understand user satisfaction by setting five evaluation items: “Excellent”, “Good”, “Neither”, “Not Very Good”, and “Bad” for evaluating some of the features available using the extension. In this evaluation, we focus on four primary features of the extension listed in Section 2.3: “List of assignments and test&quizzes”, “Coloring of course tabs”, and “Notification of new assignments”. The objective of this question is to gauge users' opinions on the available features and their level of satisfaction, providing insights on future enhancements.

We then analyzed the survey results for each feature. The scores for each evaluation criterion are shown in Table 5, and based on these scores, we calculate the average satisfaction index for each feature. The average satisfaction index is calculated by multiplying the number of respondents by the score for each evaluation criterion and then dividing the total by the number of respondents.

The results for each feature are shown in Fig. 11. The average score for the list of assignments and test&quizzes was the highest at 4.82. The breakdown of evaluation categories was as follows: “Excellent” accounted for 85.4% of respondents, “Good” accounted for 11.7%, and “Neither” accounted for 2.9%. No respondents answered “Not Very Good” or “Bad”. Given that over 90% of users utilized this function in the previous section, it is considered that such a high satisfaction rate was achieved. The average score for the coloring of course tabs was the second highest at 4.72, following the assignments and test&quizzes list. The breakdown of evaluation categories revealed that 76.0% of respondents rated it as “Excellent” and 20.4% as “Good”, with positive opinions accounting for 96.4% of responses. Since these two features were implemented with the intention of improving the usability of the LMS, it is encouraging to see that they are being utilized by many students and that they are highly rated. The notification badge feature was rated as “Excellent” by 46.3% of respondents and “Good” by 37.8% of respondents, with positive opinions accounting for 84.1% of responses. Comparing this result with the previous two features, it can be seen that the notification badge feature is not as highly rated as the other two features, but it is still rated positively by a majority of respondents. This majority of positive opinions suggests that the notification badge feature contributed to the prevention of missing assignments and test&quizzes and to the ease of noticing newly added assignments and test&quizzes.

In both Fig. 10 and Fig. 11, it is clear that features such as the assignment and quiz lists, along with the coloring of course tabs, are particularly valued. These features are preferred since they allow users to manage their assignments more efficiently without the need to navigate to each course page. The survey results suggest that features designed to streamline task management are greatly valued by users. This underscores the importance of developing functionality that enhance task management efficiency and effectiveness, which is crucial for improving the usability of the LMS. Future enhancements to the LMS requires consideration of these user preferences to ensure that the LMS continues to meet the needs of students.

5.4　Survey Results on Other Aspects

Subsequently, a survey was conducted regarding aspects of the extension other than its main features. This survey sought to determine how the official recognition of the extension by the university influenced users' perceptions and to assess whether users hope for the extension to continue being developed by students even after the current developers graduate.

The survey responses were collected in multiple choices, with options being “Strongly Agree”, “Agree”, “Neither”, “Disagree”, and “Strongly Disagree”. Survey questions are shown in Table 6.

The results regarding the official recognition of the extension show that 41.9% of respondents “Strongly Agree”, 31.3% “Agree”, 25.9% “Neither”, 0.5% “Disagree”, and 0.5% “Strongly Disagree”. The number of respondents who answered “Strongly Agree” and “Agree” accounted for 80% of the total, indicating that the official recognition of the extension positively impacted the user's perception of the extension. However, despite the extension's official recognition, 25.9% of respondents answered “Neither,” suggesting that a subset of students may be indifferent to formal recognition, focusing primarily on the extension's functionality. Nonetheless, these findings underscore that official recognition from the university is crucial, not only for enhancing the majority's confidence in the extension but also for fostering a supportive environment for student-led development.

Regarding the continuation of student-led development, 80.6% of respondents “Strongly Agree”, 16.2% “Agree”, 3.2% “Neither”, and nobody answered “Disagree” or “Strongly Disagree”. The result reveals that an overwhelming majority of respondents, comprising 96.8% of the total, express a desire to continue student-led development even after the current developers graduate. The future of the extension's development remains uncertain after the graduation of its original developers, including the first author. As the extension is being developed as open-source software, other students could continue development after the original developers' graduation. Therefore, it is our hope that the extension will continue to be developed by students even after the original developers graduate.

5.5　Overall Evaluation of the Extension

Lastly, we conducted a survey to evaluate overall satisfaction with the extension. The survey question is shown in Table 7. This question was designed to evaluate the overall user experience, encompassing all the features available by installing the extension. Respondents were asked to select one of five evaluation categories: “Very Satisfied”, “Satisfied”, “Neither”, “Not Very Satisfied”, and “Unsatisfied”.

In question 6, evaluating overall satisfaction, 85.4% were “Very Satisfied”, 13.0% were “Satisfied”, 0.9% were “Neither”, 0.7% were “Not Very Satisfied”, and 0% were “Not Satisfied”. This result indicates that the majority of users, accounting for 98.4% of the total, were satisfied with Comfortable PandA and that the extension effectively enhanced the usability of the original LMS. This suggests that students have received the extension well and contributed to reducing the burden on students in the unprecedented scenario of online classes using the LMS during the pandemic.

6.　Concluding Remarks

This paper introduced Comfortable PandA extension, an enhancement to the LMS at Kyoto University, and presented its implemented features and feedback from students who utilized the extension. Before the pandemic, the LMS was not widely utilized; thus, its convenience was rarely discussed. However, with the transition to online courses during the pandemic, the LMS has become an essential tool for faculty and students and was utilized across various courses. It became apparent that the LMS was not well designed for students who take multiple courses and that its usability was a significant issue for students. The developed extension has successfully addressed these usability issues from a student's perspective, substantially improving the user experience of the LMS. As a result, it has reduced the burden on students of checking assignments and test&quizzes of multiple courses one by one. It has contributed to reducing the stress on students in the unprecedented scenario of online classes using the LMS during the COVID-19 pandemic. Notably, the feature that colors items according to their deadlines is unique to our extension and has received positive feedback from students. To the best of our knowledge, no other LMS in Table 1 offers similar functionality, underscoring potential value of our findings for the future improvement of LMS platforms.

The number of students using the extension has been growing daily, and currently, approximately 75% of undergraduate students at Kyoto University are utilizing the extension. A significant factor behind the increase in users is the collaboration and official recognition from Kyoto University, which has reassured students of the extension's reliability. Furthermore, we have internationalized the extension, creating “Comfortable Sakai”, to make it compatible with other universities that adopt the Sakai LMS. Consequently, Kyoto University and several other higher educational institutions have started utilizing the extension.

Significant challenges ahead include the sustainability of the extension and the establishment of a system for incorporating LMS enhancements from students. As for the developed extension's sustainability, the primary development and maintenance team comprises graduate students who will graduate from Kyoto University in a few years. Finding students to take over the development of Comfortable PandA, address issues related to updates in the LMS, and fix any other issues is an immediate concern. However, since the extension is developed as an OSS project on GitHub, anyone can freely copy and modify the source code to maintain the extension. Therefore, it is anticipated that the extension can be maintained by voluntary students even after the original developers' graduation.

Another critical aspect is the establishment of a system to accept LMS enhancements from students, which is important for both Kyoto University and its students. Following the development of Comfortable PandA, Kyoto University has already established a set of guidelines called Code for PandA. These guidelines support an environment where students can lead in developing and publishing software to enhance the LMS. This collaborative approach is expected to improve the learning environment at the university incorporating students' perspectives. Nevertheless, careful consideration is needed on how to officially accept student-developed software and inspect its source code for potential security risks in the future. During the development of Comfortable PandA, the first author had a chance to have several discussions with the second author, who was responsible for operating and maintaining the LMS at IIMC, Kyoto University. These discussions, which included feedback on the functionalities of the extension, played a crucial role in its first official recognition by the university. However, such opportunities for direct communication between a student and IIMC are not always available, highlighting the need for a formal system to accept and evaluate student-developed software in the future.

Despite these challenges, integrating students' perspectives into the LMS enhancements is becoming increasingly indispensable for its future evolution and continued improvement in the learning environment.

Acknowledgments This work was partially supported by JSPS KAKENHI Grant Number 20H00099 and 24H00164.