RSChallengeEval

This repository contains the official evaluation for the ACM RecSys Challenge 2014.

The program calculates normalized Discounted Cumulative Gain at 10 (nDCG@10) given an input of ranking predictions and true rankings. Note that the evaluator assumes all data files to be in comma-separated-value (CSV) format.

Running the evaluator as a standalone program

The evaluator is a Java program. To run it, download the compiled jar and execute it like:

java -jar rscevaluator-0.14-SNAPSHOT-jar-with-dependencies.jar "/path/to/testfile.dat" "/path/to/predictions.dat"

where testfile.dat is the testset provided by the organizers and predictions.dat is the predictions generated by you.

Using the evaluator from your Java program

You will need to compile your program with the evaluator in the classpath. Compiling:

javac -cp ./rscevaluator-0.14-jar-with-dependencies.jar Example.java

Executing:

java -cp ./rscevaluator-0.14-jar-with-dependencies.jar:. Example

An example class using the evaluator:

import com.recsyschallenge.evaluate.Evaluator;
import java.io.IOException;

public class Example{
	public static void main(String args[]){
		try {
			Evaluator evaluator = new Evaluator("/path/to/testset.dat", "/path/to/predictions.dat");
			System.out.println(evaluator.evaluate());
		} catch (IOException e){
			System.out.println("Incorrect file paths.");
		}
	}
}

Building jar with dependencies

Note that this step is not necessary, it is only provided for those that want to extend the evaluator to include other metrics etc.

Should you want to build the evaluator jar yourself, you will need:

• Maven

To build a jar (with dependencies), execute the following command:

mvn assembly:assembly -DdescriptorId=jar-with-dependencies

You can also run the file program directly with Maven. In order to do so, execute the following command:

mvn exec:java -Dexec.args="/path/to/testfile.dat /path/to/predictions.dat"