This winter semester I have been co-teaching two courses: Environmental Economics and Non-market valuation. As of last week, the teaching has officially ended. While teaching a course was a very rewarding experience, now comes the time that most students dread: exam season.
Our task was to create the exam for our students. As usual, I like to explore my alternatives to minimize the amount of work in the long-run. I was investigating how to create a take-home exam that could be easily replicated next semester. I found a possible answer in the R package called R-exams. The package was created by a team of researchers from several different universities, and it is maintained by Achim Zeileis.
The package is called “exams” in R, and I can install it and use it after calling the usual library function:
Creating an exam is straighforward. All that is needed is to: 1) define a working directory where exercise files are saved:
2) create a vector (which I called “exam”) including all the exercise names:
exam <- c("Exercise 1.Rmd", "Exercise 2.Rmd", "exercise 3.Rmd")
3) Use one or more exams2(…) function. For example, to create a PDF file, I use the exams2pdf function:
The exam PDF should appear in the your working directory.
Most of the work is not using these functions, but rather creating the exercises. In the example above, the exercises were “Exercise 1.Rmd”, “Exercise 2.Rmd”, and “Exercise 3.Rmd”. No R package will decrease the workload from having to create these exercises. Many examples of exercises and how to write them in .Rmd format are available at the official R-exams website.
As for the exercises themselves, the R-exams package can compile several questions of different types. These include numeric, string, single choice, multiple choice or multi-part questions including a combination of the types above (what the package calls “cloze” questions). The .Rmd file is where the type of question is defined: “extype: cloze” or “extype: schoice”.
There is plenty of useful information at the official R-exams website to help with both compiling the exam and generate exercises: http://www.r-exams.org/. The website includes many written and video tutorials about how to get started with R-exams, as well as plenty of exercise examples. In my opinion, it was easy to get started with this package given the abundance of examples from which to learn.
There are advantages and disadvantages to using this package to generate exams. Our university uses OpenOLAT, which is a learning platform where we upload course material. OpenOLAT has the option of adding an exam as a course component, so the students can take the exam from home. The nice feature of R-exams is that I was able to generate a whole exam with R code and then simply import it into OpenOLAT. I would have to make minor adjustment to the exam afterwards.
The super feature of the R-exams package, in my opinion, is its ability to generate random numbers within exam exercises. Due to the current pandemic, all exams needed to be converted into take-home exams. To discourage cheating, we were looking for an alternative to randomize parameters within the exam. Just like R code, the R-exams package can generate random numbers from a pre-defined interval and insert them into the exam text. An example about how to introduce random numbers in the R-exams packge is available here: http://www.r-exams.org/templates/fruit/.
However, I opted against using a R-exams-generated exam for my course. There are several reasons for this, some of which are related to the limitations of the package and some are related to OpenOLAT. I will skip discussing the later in this blog post.
The main reason I was disatisfied with this package was the limited type of questions that it allows for (especially that are supported for OpenOLAT). In many instances, we would like to give students the option to either explain their reasoning in an essay-type question, or show the different calculations performed to get to the right solution. For this, we would need a large space to write text, or the option to upload other documents where students show their calculations to get the final answer. All of these options are available when constructing an exam directly in OpenOLAT, but not available when creating the exam with R-exams. The most it can do is to create a string-type question with limited space to type the answer, and then change it slightly in OpenOLAT. This would defeat the purpose of programming the exam, since everytime I would change the R code, I would have to customize the exam in OpenOLAT.
One option we considered was to create numeric exercises in the R-exams, where the parameters are randomized, and import those questions into OpenOLAT. We could then complement the exam with other type of questions and customize our exam without the limitations of OpenOLAT. We opted against this option since we weren’t quite sure how the exam would look in practice. Moreover, the problem of not allowance for large text boxes for students to answer would persist.
Overall, I found the package to be quite interesting. Its main advantages are perhaps not to create exams, but rather create smaller tests for students to access their knowledge throughout the course. I felt that this package might be especially relevant for questions that can be easily graded (e.g. single choice).
Grün B, Zeileis A (2009). “Automatic Generation of Exams in R.” Journal of Statistical Software, 29(10), 1–14. doi: 10.18637/jss.v029.i10.
Zeileis A, Umlauf N, Leisch F (2014). “Flexible Generation of E-Learning Exams in R: Moodle Quizzes, OLAT Assessments, and Beyond.” Journal of Statistical Software, 58(1), 1–36. doi: 10.18637/jss.v058.i01.