Submission Instructions

All sequential tracks will be run on the StarExec-cluster. The solvers will be executed with a time limit of 5000 seconds and memory limit of 128GB. The StarExec system will be also used for submitting the solvers.

Usage of StarExec

Getting an acount

  1. Register for an account in the SAT community (unless you are already registered).
  2. Wait until one of the SAT community leaders approves your request (should not take more than 24h).
  3. If you register late in the season, it may also be necessary to add your new user account to the respective spaces mentioned below. Please send an email to the organiziers if you do not seem to be able to find the necessary spaces on StarExec. (Also note that you might need to log out of StarExec and then log in again to see such changes taking effect.)

Uploading a solver for testing

  1. The solver is submitted by uploading its source code and a build script. The solvers are then compiled by StarExec.
  2. The uploaded file must have a special format (see the StarExec User Guide for details, here is an example with minisat). Please follow the required directory structure precisely otherwise StarExec will not be able to compile and run your solver.
  3. Login to StarExec and click Spaces->Explore (on the top) then on the left open root->SAT->Sat2023Testing
  4. At the bottom of page click "upload solver" to open the upload dialog.
  5. Click "Browse" and select the zip file with your solvers code, enter the solver name, change downloadable to "no" (otherwise everyone can see you source code) and click "upload".

Testing your solver

We put a few testing instances in the Sat2023Testing space which you can use if you uploaded your solver to this space. To run a test:

  1. Go to the root->SAT->Sat2023Testing space.
  2. Unfold "n jobs (+)" and click "create job". This opens a page where you don't need to change anything so just click "next". Use the preprocessor "xcat" to decompress benchmarks (even if your solver can decompress).
  3. On the next page select the "choose" option (for the solver), on the next page select the "all" option (for the testing instances)
  4. A page with all solvers (and their configurations) comes up, select the solver you want to run and click "submit" at the bottom of the page.
  5. You can check out the status and results of your test job by clicking on Jobs on the top and selecting your job.

Outputing proofs in the Main Track

The proof for UNSAT instances must be written in a file called proof.out and placed in the folder which your start script gets as the second parameter ($2). Thefore your starexec_run_default should contain something like 

#!/bin/bash
		./solver $1 --proof-file=$2/proof.out

Note, that the answer line ("s SATISFIABLE" and "s UNSATISFIABLE") and the solution in case of a satisfiable instance (lines starting with "v") must still go to stdout (not into "proof.out").

Testing proof output validation

To test that your proofs are correctly validated by the proof checker of your choice, you can use the script starexec.py to generate a job xml-file for StarExec. The script takes the following arguments: 

		./starexec.py test_$solver -p $pipeline -s $solver_id -b 15861098 15861099 15861100 15861101 15861102 15861103 > test.xml
Here, $solver is the name of your solver, $pipeline is the name of the proof-checker pipeline you want to use, and $solver_id is the id of your solver configuration as found on StarExec (example here). For $pipeline you can insert any of the following strings: Zip the resulting test.xml and upload the result to the testing space Sat2023Testing using Jobs/"upload job xml" button. This will run your solver on the six testing benchmarks (15861098, 15861099, 15861100, 15861101, 15861102, and 15861103) out of which three are unsatisfiable. When inspecting the finished job, do not forget to check both "job information" and (the more detailed) "job output".

Uploading the final version of your solver

The submission of the final version of a solver is essentially the same as uploading the solver for testing (see above). To submit the final version of your solver upload it to the space root->SAT->SatComp23.

Don't forget to select "downloadable:no" when uploading your solver to prevent other participants from seeing your source code before the start of the competition. After the solver submission deadline we will ask all the participants to update the downloadable option to "downloadable:yes". This will make the source code visible to the organizers and all the participants. The participants must comply with this request in order to participate in the competition (except for the No Limits Track).

Emailing your System Description Document and Benchmarks to the organizers

After you have submitted the final version of your solver send an email to the organizers (organizers@satcompetition.org) containing your system description document and 20 benchmark problems (see General Rules for more information).