Documentation
  1. Introduction
  2. User Introduction
  3. Exercises
  4. Test Data
  5. Settings
  6. File-based exercises
  7. Execution Environment
  8. API Reference
  9. Release Notes
Quick Start Guides
  1. How to set up your first exercise
  2. How to set up a file-based exercise
  3. How to use generators
  4. How to create test data for my language?

How to set up your first exercise

Introduction

In the following we will show how to create an exercise and set it up in CodeJudge. We will use Python as language, but it can easily be substituted by another language (see How to create test data for my language?). In this guide, we will go through the following steps:

  1. Create an exercise
  2. Prepare a solution
  3. Prepare test cases
  4. Upload to CodeJudge
  5. Test it!

Please note that this is a quick start guide, and CodeJudge supports many more features for testing than described here. But this guide will take you through the most common features.

Create an exercise

First we should figure out what the exercise should be about. In this guide we will use the following problem:

Create a Python program that have the functions sum(a, b) that should return the sum of the numbers a and b. Furthermore it should have the function factorial(a) that must return 1 * 2 * ... * a = a! (should work for a >= 0).

  • The problem should be testable, i.e. input/output must be well defined.
  • Clearly state what classes/functions/etc. the students should make and what the indended behaviour is.
  • Mark the exercise with "CodeJudge" or something similar, to indicate the exercise can be tested on CodeJudge.
  • For more complex problems, it can be a good idea to include samples of how the methods could be called and what they should return.

Prepare a solution

Next, we'll create a solution to the problem, let us call it solution.py. The solution will serve multiple purposes:

def sum(a, b):
    return a + b

def factorial(a):
    if a <= 1:
        return 1

    return factorial(a - 1) * a
  • Use the coding style you want your students to use, i.e. remember to do indentation, commenting, etc. since it will be used as a suggested solution.
  • Spend some extra time to verify the solution is correct - otherwise, you will probably get some annoyed students.

Prepare test cases

Let us prepare some test cases, we want to to test submissions on. Each line represent an individual test.

print(sum(3, 6))
print(sum(-5, -9))
print(sum(33, 64))
print(factorial(5))
print(factorial(10))
print(factorial(2))
print(factorial(1))
print(factorial(0))

You might now be thinking that we also need to prepare what the answer/expected output of our test cases are. However, in this example we will let CodeJudge produce the expected output by uploading just the "input" of the test cases and the solution. (It is possible to manually specify the expected output instead - see the documentation for this)

Test cases can be much more complex than this example; they may involve input from the console, command line arguments, files, multiple correct outputs, etc. For more about this, see the documentation.

  • Put common case tests first, then special cases and finally larger tests. This will make debugging easier for the students.
  • Be thorough with the test cases, students sometimes make very unexpected solutions. In our experience 5-15 test cases will be sufficient for most problems - obviously depending on the difficulty.
  • Try not to make unnecessary long and complex test code, remember the students should be able to understand what is tested.
  • When creating tests for special cases, make sure the correct behaviour is actually described in the problem text.
  • You can test error handling with "invalid" input by surrounding test code with a try-catch block (again, remember to tell students the program are supposed to throw an exception in this case).

Upload to CodeJudge

It is now time to setup the exercise on CodeJudge, so we will navigate to our course on CodeJudge. Before setting up the exercise, we have to create an exercise group. Normally one will create an exercise group per week/exercise class/similar. Follow these steps:

  1. Select "Exercises" in the menu.
  2. Select "Edit" in the page tabs.
  3. Click "New Exercise Group".
  4. Give the exercise group a name (like "Week XX") and set when the students should be able to see the test group in "Visible From".
  5. Click "Create".

We are now ready to setup the actual exercise.

  1. Navigate to "Edit"-tab of the exercise group you just created. (You should already be here after creating the exercise group.)
  2. Click "New Exercise".
  3. Fill in the name of the exercise and the description
  4. If Python is not the default language for your course, you should select this under "Languages".
  5. Select the file "solution.py" under "Solution/Solution Files".
  6. Select when the students should be able to see the suggested solution: always, after they have solved the exericse, or never.

Finally create the tests. This will automatically generate the expected output as well. An exercise can consist of multiple test groups, but simple exercises only need one. See the documentation for more information:

  1. (Setup a new test group - this is automatically done for you)
  2. Give the test group a name, e.g. "Tests".
  3. Select if the students should get full or limited feedback.
  4. Click on the column selector in the upper right corner of the tests table. Untick In and tick Python instead.
  5. Click "Add test".
  6. Insert the first test in the Python field (ie. "print(sum(3, 6))").
  7. Repeat step 5 and 6 for each test.
  8. Finally, click "Create" in the bottom to create the new exercise.

After a short while of generating answer files, the exercise is ready!

Test it!

As a final step, it is a good idea to see if the exercise actually works as expected (especially when it is the first time you create one). Navigate to the "View"-tab of the exercise and try to upload your solution. If you have not tried CodeJudge before, it is also a good idea to experiment with the different kind of mistakes students might make (compiler errors, programs producing exceptions, wrong output, etc).

Managing many exercises?

If you are going to manage many exercises, it might be tedius to use the web interface for this. Instead we support file-based exercise management. See the guide How to set up a file-based exercise to learn how to do this.