MuSe Submission

Scoring test partitions

Formatting requirements

It is mandatory that your submitted predictions comply with format described below. If not, you will receive a format error email. You can upload predictions for one sub-challenge or for all at once. The name of every prediction file must be 'test.csv'.

The directory 'label_segments' includes examples of the aggregated labels for each sub-challenge.

  • The structure of the prediction files has to be the same EXCEPT that the column names starting with "label_*" are replaced by "prediction_*).

  • The files has to be NaN free and include all timesteps to predict (see masked 'test.csv' files)

  • The format has to be a comma-separated '.csv'.

  • The name of the prediction file has to be 'test.csv' (example below).

We use the aggregated files (of the not distributed test set) as well as the submitted predictions to calculate your scores. The scoring script will be released with the baseline models on our github that you can test beforehand if your files are correctly formatted before you submit them officially.

Examples

MuSe-Wild

Example: test.csv

id,timestamp,prediction_arousal,prediction_valence

23,250,-0.9896,-0.3857

23,500,-0.9881,-0.3909

....

MuSe-Target

Example: test.csv

id,segment_id,prediction_arousal,prediction_valence,prediction_topic

23,1,0,0,6

23,2,0,0,7

...

MuSe-Trust

Example: test.csv

id,timestamp,prediction_trustworthiness

23,250,-0.4288

23,500,-0.4268

...

Submit here:

The number of submissions of test results is limited to five trials per team and Sub-challenge.

You have to be logged in with a Google account in order to see the form above (otherwise it is greyed out).

If you do not have or do not want a google account, you can also submit the predictions also to contact.muse2020@gmail.com.
Processing may take longer.

Public Leaderboard Policy

If you want that your team score appears on the public leaderboard, please reply to the email with the scored predictions (form above) including

* name of team members

* a link to a Github repository where their solution/source code is uploaded for replication of results

* a link to an ArXiv paper with 2-6 pages describing their proposed methodology, data used and results