Submission Guidelines¶
Preparing for Submission (Validation stages and Final submission)¶
Methods can only be submitted as a Docker. To prepare your Docker, please follow our instructions on our Preparing Docker page. Both the Docker image and Dockerfile should be sent to us using the SurfFileSender voucher (please ask us for a voucher when you are ready to submit). See further instructions below on how to prepare your Docker
Submitting your docker¶
- Send us your exported Docker (TeamName.tar.gz) as well
as the corresponding DockerFile with surffilesender
- Please request a surffilesender voucher by sending us an e-mail (if you already have a surffilesender account, you can just send it with your own account)
- We will run your Docker on a case of the training set and send it to you so that you can check it is what you expected. Please let us know if your method is working as it should or if there are any discrepancies.
Please use the following naming conventions:
- For your Docker tag: TeamName:latest (e.g. findvaldo:latest if your teamname is "findvaldo")
- For your exported Docker: TeamName.tar.gz
Validation and submission timelines¶
In addition to the final submission (by the 5th August 2021), you will be allowed to submit and assess your proposed Docker solution in two validation stages during which you will be able to submit ONCE a Docker to assess.
- The first validation period occurs between the 14th and the 21st June. You should hear about the results about a week later
- The second validation period will occur between the 12th July and the 19th July
The validation set is composed of a set of 5 images for each task chosen from the RSS dataset. This set does not overlap with the training set nor the test set.
The final submission period will start on the 22nd July and close on the 5th August.
Further important remarks¶
The Docker image will be run without access to internet.
These instructions apply to the submission for the evaluation moments (on the validation set) as well as for the final submission (on the test set).
If your Docker needs a GPU to run, please make sure your Docker is
compatible with the Ampere GPU architecture. If you use a relatively new
version of TensorFlow/PyTorch it will probably be compatible, but please
check to be sure . The PyTorch version should support CUDA capability
sm_86. The CUDA version should be >= 11.0.
If this is an issue,
please reach
out to us and we can discuss the options.
See the challenge design document for more information about our challenge.
Final Submission¶
During the final submission stage, you will be able to submit your method ONCE. We won't accept more than one submission per team. If you don't have a SurfFileSender access, please do email us so that we can send you a voucher for your docker and Dockerfile.
When submitting your proposed method for your final submission, you will be asked to fill in a questionnaire and upload a pdf of 1 to 2 page describing your method and your initial results.
Submission questionnaire for
The document to prepare should contain details about
- Any preprocessing performed on the images
- The type of method used (e.g architecture of deep learning network)
- Details on the implementation (e.g learning rate, stopping criteria, intensity threshold...)
- Any initial results obtained on the training set