In light of the recent controversy regarding the new KLMS, The KAIST Herald interviewed two employees, Bumhee Lim and Kyungduk Kang, of the Learning & Teaching Support Team at KAIST. Bumhee Lim greeted us, stating in advance that there was much to discuss. The interview was two hours long, and includes explanation of why and how the new KLMS was developed, why some features were designed this way, personal hardships, the future of KAIST online education, and her hopes from students and the school administration. Here is the full interview:
Please introduce yourself to our readers.
My name is Bumhee Lim, and I am the technician for the Learning & Teaching Support Team, part of the Center for Excellence in Learning and Teaching (CELT). I originally worked on establishing KAIST’s academic system for ten years, and it has been three years since I was transferred to this team.
Why and how did the recent KLMS renovation take place?
When I first came to this team, there was a basic learning management system (LMS) in place. The very first LMS was created in 2011, when students questioned why KAIST did not have its own LMS system and created one themselves. The students used Moodle, an open-source learning platform, and created an LMS with the address of moodle.kaist.ac.kr. Then, when the former President Nam-pyo Suh came into office, he emphasized the need to innovate KAIST’s education system; therefore, a budget was allocated and the work on a newer LMS began. There was also an order to place “K” in front of the LMS, and thus KLMS first opened in 2013.
As you can see, the server of KLMS was created by the students in 2011. The lifespan of a server is only six years. Because back then we had fewer number of courses, there was not a major problem to run the system. Of course, there were problems such as slow speed, and inconveniences from the fact that the system was Flash-based. When I first walked into the server room, the servers were, what should I call it… ragged.
There were 39 machines — most of which were ones others threw away — running the entire system. I wanted to reorganize the servers when I first transferred into the team [in 2018], and it wasn’t an easy situation. But then COVID-19 pandemic happened. I was not at all prepared."
When I came to the team in 2018, there were problems already related to the system falling apart. So, we ran a student monitoring team. For a few semesters, we recruited seven students (one international student, one freshman student, as well as PhD students and TAs). For six semesters, we ran various surveys and interviews regarding problems of the LMS and potential new features that could be added. Then, at the end of the 2020 Spring semester, we collected a survey on student satisfaction on online classes via a pop-up on KLMS. At that time, thankfully, the largest-recorded number of over 600 students responded to our survey. There were over 2,300 recommended features, and we narrowed them down to 100 features which the students most wanted. We visited the Vice President, Seung-seob Lee, and received 150 million KRW to establish the system.
To tell you the truth, 150 million KRW is not a lot of money; it is approximately enough to build just the homepage. To build an entire LMS, other universities have allocated up to 500 million to 1 billion KRW on the project."
So we split all the budget we had and freed up about 350 million KRW. Using these funds, we had to renew the old server infrastructure, and throw away the old LMS because the LMS is Flash-based, and as you know, Flash went out of service at the end of last year. Furthermore, the school had other requests, such as creating a virtual campus platform to collaborate with other universities and creating a military credit certification system. But I was the only one running the system, the only one building a new system, and the only one in charge of the server and security of the system. It was very challenging to respond to COVID-19 and to develop a new system by myself. I think last year was the toughest year personally of the 20 years that I worked at KAIST.
How does KLMS compare to the LMS of other universities?
For three years, I observed the LMS of other universities in order to develop our own. Out of all the cases, Sungkyunkwan University and Chung-Ang University had the best implementations and a tremendous budget. I was very envious of the fact that Sungkyunkwan University had nearly 30 employees in its education center. They had a separate AI team, a big data team, education and learning analytics team, and an operations team. I was very jealous — it was me alone here. Many universities have a separate task force specializing in managing their LMS system, but here, we do not.
As I was researching the cases, looking at the organization chart and what LMS solutions universities were using, I found that there were three major solutions: Blackboard, Moodle, and Campus. Seoul National University, Yonsei University, Ewha Womans University, KAIST, and GIST used Moodle; Korea University and Hanyang University used Blackboard; and Chung-Ang University and Sungkyunkwan University were using an open-source software called Campus.
Why was Zoom integrated into the LMS?
Have you heard of the Minerva Schools? We were given an order from the President [Sung-chul Shin] to create a system like the Minerva Schools. The most important aspect was active learning. When we observe Zoom classes, many students are wearing a mask, somewhat hiding, or turning off their camera completely. Active learning is a tool that can record the student’s face, activity, motion, as well as audio. A data log of a student’s learning is created, and a green or red card is displayed on top of the student’s head using that data. If the student works hard, it’s a red card; if the student does not participate with their heads all down, it’s a green card. The professor can then ask questions to the students with a green card. Active learning is a system where technology provides the tools for such interactions.
This was the reason why we integrated Zoom in the new KLMS. Some students have said that it is rather inconvenient and that they would prefer to enter the Zoom meeting via a link and not via KLMS. However, Zoom, just like Minerva Schools, collects lots of data — not just attendance, but also things like how much a student talked during the meeting. We can collect all this data and use it for future analysis and study. Actually, foreign universities already integrate the Zoom API and use it for learning analytics data. Our plan for the next three years is to collect the log data with Zoom and advance the LMS version to version 3.9. We can analyze how often students review, when students enter KLMS, whether students submit homework once or multiple times, etc., and analyze the patterns of not just the students themselves but also that of the course itself.
Was there a beta, or a monitoring of the new KLMS before the actual release?
When we were developing the new KLMS, we ran it with five students as part of the monitoring team. For instance, we asked for their opinion by screensharing the new KLMS with them, and asked them to ask people they know for opinions as well. The majority of students probably first saw the new KLMS without knowing why it was designed this way — in reality we developed the system with student monitoring taking place. Furthermore, before the new KLMS officially opened, we conducted a trial run of the new KLMS on the classes of Professor Mik Fanguy and Professor Seon Min Park [from the School of Humanities and Social Sciences], who were the best at utilizing the LMS system.
How did you improve the servers?
As I said previously, the server equipment was all decrepit; for some, when they turned off, they did not even turn on again. It costs a lot of money to completely overhaul the equipment, so we instead launched everything into NHN Cloud service, which is used by public institutions. One advantage of the cloud is that the National Intelligence Service protects the data. So all of the websites students access are no longer actually run on school equipment but the cloud.
The Server Shutdown of March 2
On March 2 (the first day of this semester), about 4,800 students logged on the system simultaneously. Two web servers cannot handle such traffic, so the system shut down for about three hours. If we had continued using the school server, we would have had to buy new servers and add them to the system, which would have meant the system being shut down for a few days. But because we are now cloud-based, we hurried to add three web servers in three hours, making a total of five web servers.
Some have been asking why we didn’t predict this would happen. But we were just barely setting up the servers — as you know we lack the funds in this area. It costs nearly 10 million KRW monthly to use the cloud service. If we increase the number of web servers to five, our budget would not be able to cover that. So, we quickly cooperated with the budget team and purchased the servers on credit.
I think that it is sad to even think of shutting down the system because we lack funding in a scholarly institution where learning must be placed first and foremost... I wish the school would pay more attention and invest in us with a little more budget."
If so, something embarrassing like KAIST asking to borrow three web servers because it lacks the budget would not happen again in the future. Thankfully, the team at NHN generously offered us the web servers for free and the situation improved.
International Students taking online classes abroad
Our school still has students from over 60 countries worldwide taking classes from abroad. Previously, with the old KLMS system, students living in countries with poor network connection could not take classes because the signals could simply not reach the servers in South Korea. So unfortunately, many international students could not take classes in 2020 Spring.
This was a major problem. So, from last fall, we used a contents delivery network (CDN). Using CDN, if the professor uploads a video on KLMS, a cache is submitted to cache servers in 133 countries worldwide. For example, if you are in London, you are watching the videos from the server based in London. It was more expensive to implement, but we worked to develop the system so that students living abroad could also watch the lecture videos on KLMS.
How did international students abroad take classes before the CDN was implemented?
There were three options. First, unfortunately, many students took a leave of absence. Second, there were alternate ways of grading, such as through assignments, or alternatives to original assignments. Third, some lectures were uploaded to YouTube, and some lectures were compressed to mp4 files and mailed to these students.
Currently, content downloading is not possible unless the professor or TA allowed it. Why was such a change introduced?
I had a lot of trouble with this aspect. Last year, I was frequently called on by the legal team regarding copyright issues. They asked me why downloading of web content was enabled, and asked to create a viewer to display the content instead to resolve copyright concerns. It was very tiresome.
So, when I integrated a web viewer and made it as default, students started asking why downloading did not work and the bulletin board was all over the place. I didn’t know who to listen to!"
I will change the system today (March 9, day of interview) even if I have to stay up the whole night. I will change it so that the default is an empty value, which means professors must choose between allowing downloading and the web viewer. If the content the professor is about to upload is protected by copyright, he or she should choose the web viewer; if the content is without copyright, he or she should allow downloading. As of now, many professors are saying they didn’t see the message, but now there will be a notice displayed so that it will be impossible not to see the message. I presume that in about 80-90% of the cases, professors will allow downloading of the content.
Why was the lecture rewinding (when students watch for the first time) made not possible?
This was the guideline set by the Ministry of Education. They say that if you rewind and fast forward, for instance, a 30-minute lecture, there is no way to confirm whether you listened to it entirely. So, I ask the students to please watch the lecture regularly for the first time for the sake of progress checking, and then freely rewind or fast forward lectures from the second time, which was made possible.
Regarding the mobile app, when will the iOS version be released?
The previous KLMS mobile app was not our implementation but was one developed and run by another company. But that company went out of business, so the system became very unstable. Therefore, we got rid of the previous KLMS app and released the new Android application first. We actually applied for the iOS version on the same day, but it was rejected three times. When we asked why, we were given answers along the lines of the app not having the correct screen size. We will work to improve the app and release its iOS version in the near future.
Are there plans to improve KLMS in the future?
When the spring semester ends, we will conduct another survey, including the positive and negative aspects of the new KLMS, as well as features that should be improved. Then we will develop the KLMS one more time based on the list of potential improvements.
We want to continue to upgrade the system according to the users’ suggestions and demands, so I want to ask for everyone’s understanding even if the first newly opened system is not up to their standards."
Who can students contact to raise concerns or give feedback about the new KLMS?
When students post questions on the Q&A page at KLMS, we try, if possible, to answer it the day it was posted. Then on the weekends, I collect the most frequent questions and post them on the FAQ. Moreover, I will collect the important issues within the FAQs and develop those features for the fall semester.
It would be great if students directly post their opinions and problems regarding KLMS on the Q&A page, as we will answer them and potentially apply them in the future.
Favor from the students
Last February, when COVID-19 started, I went to the server room to set the servers, answered civil complaints in my office, helped professors film lectures… it seems like it has been over a year since I barely returned home, only to change clothes and come back to work. I collapsed twice, was sent to the emergency room, had IV injections, came back, and started night shifts. I think the vice president later found about this, and, very thankfully, sent me a new employee. He is very able and intelligent. We now answer calls and develop the system together, and I don’t think I’ll be alone in the tasks anymore. Two is not a lot, but it is of much help to me compared to when I was alone. Our department would not have anyone left if people blamed someone and punished them from it. I wish many can support us as we are trying our best to stabilize and resolve problems regarding the system.
She then introduced the new employee, Kyungduk Kang.
Please introduce yourself
Kang: My name is Kyungduk Kang, part of the Learning & Teaching Support Team, and this is my first department after becoming a new employee last year. I still have many areas I need to work on and learn. Because this is my first department, I am trying to understand the underlying aspects of the university.
Please introduce the CELT team to our readers
Lim: Recently we had a change of leadership in our team. Professor Hyeon-Jeong Suk became our new director. Previously, Professor Youngsun Kwon worked tirelessly to make everything possible, such as budgeting, persuading the vice president for us to get a new employee, and persuading other professors during many hard times. Professor Kwon came to work right after he got a cataract surgery. He has a fighting spirit, working hard even when it required getting multiple injections to stay healthy. Additionally, many members of the CELT team all have masters and doctorate degrees in pedagogy. They all work until late at night to assist in filming, education, and KLMS-related tasks. Professor Kwon, Professor Suk, and team leader Younghee Jeong all work very hard — we are like one family.
Hopes from the school administration
Lim: I’m very thankful to now have Kyungduk by my side, but realistically I think we do need a few more people. Only then can we implement other public services and do big data analysis. Personally, I don’t know much about big data. I didn’t major in this area and I’m not well versed in AI. But the school has requested us to implement augmented reality, virtual reality, systems for experimentation, and AI-based systems, but it’s hard for just the two of us to study these areas and implement them at the same time.
Many say “oh, KAIST is so advanced in such education systems”, but in reality, we are busy catching up."
To give an example, Chung-Ang University has a separate AI team called the Da Vinci Institute of Learning Innovation. Because we are part of the Ministry of Science and ICT, the ministry frequently gives us tasks to do along the lines of “can KAIST pioneer a great platform and distribute it to other universities?” However, it is hard to work on those when we are managing the LMS at the same time. My wish is for our team to be composed of more young and able people to work on these important tasks in the long run.
Regarding military credit certification
Lim: From two to three years ago, the Undergraduate Association requested to create a military credit certification system. But this was faced with two major problems back then. First, there were no lecture videos that could be used for the remote classes. That means there were no available lectures in the first place. To elaborate, to allow students at the military to take classes remotely, someone needs to film all the lecture videos and create online content for those students. There were rules from the Department of Education regarding these classes, such as one session must be over 25 minutes, and that certain information must be included in those lectures. There needed to be videos filmed with those guidelines, but there were none, and there were no professors who were willing to do so as well.
But from 2020 (due to COVID-19), there were suddenly all these lectures filmed and contents available. We now have some flexibility in this regard. Our current plan is to make the military credit system possible starting from the 2021 Fall semester. We would need to do additional preparation, such as giving incentives to professors willing to take on such tasks, which means budgeting for incentives as well.
The other problem back then was that we needed a biometric authentication system, which we do have now. So basically, the system is all set, we just need the lectures. We will film the lectures with professors this year. If we look at cases of other universities such as SNU, students can take a special leave from the military to take exams at school. Things like this can differ from school to school, and the main task left is to integrate LMS to the military portal and we will be able to open the system by this fall semester.
What is your typical day at work like?
Kang: It differs from time to time, but recently when I come to work I usually test various aspects related to improving the stability of KLMS. Then at night, I study relevant aspects. There’s no exact routine but we are receiving a lot of calls so I answer those as well.
Lim: It is busy every day. When I come to work I usually check my mail and answer questions uploaded to the Q&A page. I check if the servers are running alright. Then I start answering the calls to our office. If I decide we need to fix some programs, I usually contact outside developers remotely and fix the systems together. I do have some free time during the weekends but sometimes I do answer calls forwarded to me even during this time. I hope to stabilize the system by March, so that I can rest on the weekends and leave work at 6 p.m.
Do you have any final comments for our readers?
Kang: I am very sorry to have caused a lot of inconvenience from the new KLMS. We will work our best to stabilize it as soon as possible so that there are no problems in using the system.
Lim: I want to say the same. We don’t use the system — the students and professors do. We will try our best to stabilize the system into something students and professors can use comfortably. And I wish many can still give us a word of encouragement in order for us two to carry this project right until the very end. I had thoughts that no one recognizes our efforts to develop the system, but on the other hand, it energizes me to have been interviewed so that I can tell our story. We will work hard until the very end to create a better system for all.