Quick Start Guides

Release Notes

Release notes from August 2019.


We are thrilled to announce the next major release of CodeJudge. Last year we had a major release where most of the internal structure was rewritten. For this release the focus has been on improving the experience of setting up and managing exercises, and making the system mobile friendly.

There are a lot of changes and we hope you will find them useful! Unfortunately, some of these are breaking changes and therefore you must be aware of these when setting up exercises. We have converted all old courses to the new system, and exercises can therefore be reused by exporting and importing them.

First we will give an overview of the features/changes and then dive in to some of the details. Please feel free to contact us with any kind of problems, questions, suggestions or other feedback - we are ready to help.

Overview of new features

Overview of breaking changes

The following areas have been significantly changed:

Renamed properties

For various reasons the following properties have changed names:

Migration info

We have converted all existing courses to the new exercise format, but if you have some exercises on your computer in the old file format, you will either have to export the exercises after the conversion or manually update them to the new format.

File-based Exercise Management

It is now possible to manage assigments, exercise groups, exercises and test groups entirely using files. This allow you to use tools such as git to collaborate with colleagues on the exercises. See how to setup up a file-based exercise to get started and the documentation for details.

Below you can get an idea of how file-based exercises are structured:

Here lang is any of the supported language extensions.

Per Exercise Test Files/Configuration

Test configurations and files (including working directory files/custom judge/overwrite files) can now be set for the entire exercise - meaning all test groups in the exercise will use these. This simplifies things when you have multiple test groups. One can still set test group specific configs.

An example:

You have an exercise with two test groups: Samples and BigTests. The solutions to the exercise should be compiled with an extra file (ie. using the overwrites feature), so you set this for the exercise. Then as BigTests contains some very large tests for this test group you want to increase the CPU time limit, so you set it to 2000ms only for that test group.


For complex exercises the test data is often generated by code e.g. a python script. Instead of having to run the script locally and upload/sync all the generated files, CodeJudge now supports uploading the generator code (e.g. the python script) instead when syncing.

See the generator guide to get started with generators.

Single File Tests

A couple of changes have been made to the the test data single file format, among others to be able to specify multiple test groups within a single file. See the documentation for full details of the new format and more examples.


Command prefix: The command prefix "///" has been split up into two prefixes two new prefixes:

For languages where "//" is not a comment, we also support that the first character can be "#" or "%" i.e. "#/@", "%/$" etc.


Let's see a simple example

//@ TestGroup: Samples
//@ OnFailue: Break

//@ Test: Sample
1 3
//@ Ans: 4
//@ TestGroup: Tests
//@ Test
100 200
//@ Ans: 300

In this example we create two test groups: Samples and Tests. For Samples the property OnFailure is set to Break and a test named "Sample" with input "1 3" and ans "4" is created. For the test group Tests a test named Test01 with input "100 200" and ans "300" is created. Tests are named Test01..N when a name is not specified.

Special files can now also be created:

//@ Test: FileTest
//@ wkdir/text-file.txt
Hello world

will create the file FileTest/wkdir/text-file.txt with "Hello world" as content.

For test scripts all lines before the first _//@ Test(Group) and after a //@ End command will be written to all test files, similar to the old format:

class Exercise {
  public static void main(String[] args) {
    public calculator = new Calculator();
    //@ TestGroup: Samples
    //@ Test
    System.out.println(Calculator.Sum(1, 3));
    //@ TestGroup: Tests
    //@ Test
    System.out.println(Calculator.Sum(100, 200));
    //@ End

will output

class Exercise {
  public static void main(String[] args) {
    public calculator = new Calculator();
    System.out.println(Calculator.Sum(1, 3));
class Exercise {
  public static void main(String[] args) {
    public calculator = new Calculator();
    System.out.println(Calculator.Sum(100, 200));

(Here the multiline prefix //$ could be useful if one would create a wkdir file for instance, as the class could then still be compiled.)

File name and folder structure

The single file must now be named Tests.[ext] where ext can be any extension of a test file like before. The Tests file can either be placed at exercise level (in the exercise folder) if it is specifying test groups, or in a test group folder if it is only specifying tests (like before).

See How to create test data for my language for more examples and the documentation for more details.

Test group execution

The previous Run If property that was used to decide if a test group should be graded or not has been replaced:

Old Usage: Value should be a number, 0 (Always), -1 (Never) or testGroupId (id of the test group, which it should run after if the test group succeeds)

Motivation of change: RunIf was very difficult to use during import/export as internal test group ids had to be known.

Change: It has therefore been split up in to three new properties:

Although this might seem more complex than the old solution, in most use cases one now only need to set OnTestGroupFailure for each test group to achieve the desired behavior.

Language specific overrides

Language specific overrides have been streamlined to make it simpler to use and understand. Language specific overrides should now be placed in a root folder [lang] of the Exercise/Test group. For instance, let's say you for python want to set the time limit to 2000ms and add a file to the working directory. Then you need to set the CpuTimeLimit in py/config.json and place the lib file in py/wkdir/

Only the exercise and test group properties (e.g. Name, Visibilty, Feedback) cannot be overridden for specific languages.

Online Exercise Editor

We wanted it to be possible to create an exercise in CodeJudge without having to upload any files. We have therefore added a file editor so that the name and content of all (non-binary) files can be edited inline. Furthermore we have added a table for the test data, so that you can see and edit all the individual tests without having to download/upload any files.

Check it out under the Edit-tab on any of your exercises.


We have made CodeJudge's API public so it is possible to write scripts (or other clients for that matter) to get data from or interact with CodeJudge.

CodeJudge CLI (preview)

With the new file-based exercises, it becomes very natural to manage exercises locally and syncing them to CodeJudge when ready. To ease this workflow, we have created a simple CodeJudge CLI for synchronization of exercises, so you do not need to manually upload all the files to the system.

This also integrates great with using git for managing your exercises.