# Solver Notes

## Formula Notes

The target family is `software-bmc`. The visible training formulas are
CBMC-style Tseitin encodings of bit-vector/software model-checking instances.
The BMP280 files are small, around 23k-25k variables and 94k-106k clauses,
mostly binary and ternary gate clauses with no initial units. Other CBMC files
range from 44k variables / 232k clauses to 13.8M variables / 53.6M clauses.

Repeated gate motifs appear throughout: AND-like triples, XOR/equality-like
four-clause patterns, mux/carry clauses, and long one-hot or decoded-word
constraints. The formulas are not pure 2-SAT, Horn, or XOR systems, but they
are propagation-heavy after suitable branching on circuit/control variables.
Comments are not used by the solver.

## Techniques Used

The current solver is a from-scratch C++ solver with two SAT paths:

- DIMACS parser for sanitized comment-free CNF.
- The small/medium path is the inherited watched-literal CDCL engine using
  per-clause vectors. It keeps VeriPB RUP logging enabled for UNSAT cases.
- Medium formulas with a low clause/variable ratio first run the compact
  flat-clause engine with proof logging disabled. SAT answers from this pass
  are still fully model-checked before printing, and non-SAT outcomes fall
  back to the VeriPB proof-producing path. This avoids writing large unused
  RUP logs and is faster than the older vector-clause no-proof probe on
  medium SAT instances such as `mix034` and `safe029`.
- Small BMP-style formulas now use a narrow sequential portfolio: a
  proof-producing CDCL probe capped at 5,500 conflicts, then a flat-clause SAT
  probe capped at 5,200 conflicts, then the full proof-producing fallback.
  This keeps the easy SAT/UNSAT cases on the checked proof path and cuts the
  two slow BMP SAT variants from about 2.2s to about 0.5s locally.
- The million-variable path uses a compact flat clause database: one global
  literal array, per-clause offsets/watch positions, and linked watch lists.
  This avoids the `vector<vector<int>>` and per-clause allocation overhead
  that made the huge stack/esawn formulas impractical.
- Formulas above one million variables now skip the experimental topological
  constructor and go directly to the flat solver. The topological pass did not
  solve the hard large cases and only added a full parse/scan before CDCL.
- For the large path, branch variables are selected first from variables that
  never occur as the maximum variable of a clause. In these CBMC encodings
  that is a useful structural proxy for primary/input variables, while most
  maximum variables are Tseitin outputs. This cut the stack search from
  timeout to SAT.
- Ordered stack-sized CNFs are detected by the first definitional clause having
  a small maximum variable. Those use all variables in the flat branch heap,
  which cut the ordered stack instance from about 187s to about 78s locally
  while keeping the compact model under the stdout cap. Shuffled stack CNFs
  keep the older primary-first order because all-variable branching found a
  model whose compact hitting set exceeded the cap.
- The flat path keeps an optional phase override for experiments. By default
  it uses the occurrence-polarity phase for stack-sized formulas and an
  all-negative initial phase for formulas above 10M variables; the latter
  reduced `esawn` from thousands of conflicts to 17 conflicts locally.
- A preliminary topological model constructor groups clauses by maximum
  variable and tries to justify unit/error clauses through the acyclic Tseitin
  structure. It is conservative: it only returns SAT after full CNF
  verification. On the current train split it does not solve the hard large
  cases by itself, but it is cheap and may catch simpler acyclic SAT variants.
- Watched literals for unit propagation.
- Occurrence/activity based variable ordering with VSIDS-style bumps.
- Phase saving plus a restart-time phase schedule cycling through occurrence
  polarity, all-positive, all-negative, and saved phases.
- Geometric restarts.
- VeriPB RUP logging for every learned clause; UNSAT emits a final empty RUP
  contradiction and a checked conclusion.
- SAT model construction is conservative: before exit code 10, every original
  clause is rechecked against the complete assignment.
- For huge SAT models, the solver first tries to emit a compact partial
  assignment: a consistent set of true literals that hits every original
  clause. It checks mandatory literals from clauses with exactly one true
  literal under the full assignment. If no compact assignment fits the old
  stdout-size target, the solver now falls back to printing the complete
  internally verified assignment. The current worker verifier streams solver
  stdout from disk, so large SAT models do not need to fit in memory.
- On formulas above 10M variables, the flat CDCL loop periodically checks
  whether the current partial assignment already hits every original clause.
  If so, it emits only assigned literals. This did not trigger on `esawn`, but
  it is sound and may catch related large SAT variants before all variables
  are assigned.

This is intentionally aimed at the recovered CBMC/Tseitin structure rather
than external solver integration. No SAT/SMT/PB/MIP/CP-SAT solver is bundled,
called, imported, or shelled out to.

## Local Results

On the non-huge train files (up to 526k clauses), the current build solved all
12 instances locally. The three UNSAT outputs (`safe009`, `safe027`, and
`qpr-bmp280-driver-15`) were checked successfully with
`veripb --cnf formula.cnf out.pbp`.

The large path was tested on regular uncompressed local copies of the huge
formulas, with stdout redirected to files so the model-writing cost was
included. Current local timings:

- `26_stack_cas_longest...cbmc-u2-sc2016`: SAT in 75.84s, 1.59GB RSS,
  5,350 conflicts, 12,005,752 decisions. Compact output used 7,805,432
  literals / 65,992,714 stdout bytes and was locally checked against all
  35,975,032 clauses.
- `26_stack_cas_longest...cbmc-u2`: SAT in 78.41s, 1.67GB RSS,
  1,742 conflicts, 8,769,847 decisions. Compact output used 7,805,502
  literals / 65,996,808 stdout bytes and was locally checked against all
  35,975,032 clauses.
- `esawn_uw3.debugged`: the all-negative phase finds a full SAT assignment in
  12.95s and 2.37GB RSS with 17 conflicts / 88,905 decisions. The compact
  mandatory-literal attempt still needs 13,816,352 literals / 128,272,276
  bytes, so the solver falls back to the complete checked model. The emitted
  local stdout was 128,491,738 bytes and
  `lymphosat.worker.verify.verify_sat_model_file` accepted it.

The small BMP portfolio was locally checked after the latest changes. The two
previously slow BMP SAT files `qpr-bmp280-driver-18` and
`qpr-bmp280-driver-22` now solve in about 0.55s and 0.46s respectively; the
fast BMP SAT/UNSAT files still return through the proof-producing CDCL probe.
The medium flat no-proof SAT probe solved local `mix034` in 4.84s and local
`podwr001` in 23.05s while producing no SAT proof file; true/UNSAT medium
instances remain outside the ratio guard and still produce checked proofs.

The broker verifier reads only a capped 64MiB stdout window for SAT models.
The two stack compact models are just below that cap and locally check as
partial hitting assignments. The `esawn` complete model is about 128MiB; for
the tested full assignments, clauses with exactly one true literal force almost
every variable into any partial model. The submitted solver keeps this case
conservative and returns `UNKNOWN`.

## Remaining Things To Try

- Add clause minimization and learned-clause database reduction. The current
  code keeps all learned clauses, which is acceptable on small/medium cases
  but may hurt huge instances.
- Improve the topological justifier with bounded backtracking. The greedy
  unit-justification attempt reaches late arithmetic/comparator conflicts on
  the stack instance; smarter choice repair may solve large SAT cases without
  CDCL.
- For `esawn`-sized formulas, the local group lower bound still rules out a
  small true-literal hitting model for this instance. The current solver relies
  on the worker's streaming SAT verifier and emits a full model when compact
  transport is impossible.
- The direct greedy implicant prototype in `tools/implicant_greedy.cpp`
  selected about 2.68M literals / 22MB on `esawn` but left 13.65M clauses
  uncovered and then hit sign conflicts while patching. A more promising next
  version would combine literal-coverage branching with watched-clause
  propagation and conflict repair, rather than batch-selecting signs.
- `tools/implicant_lower_bound.cpp` groups clauses by maximum variable and
  checks whether each group can be hit without selecting the group output. On
  `esawn` it proves at least 12,521,872 output variables are required in any
  explicit partial model, already about 103-115MB before normal line overhead.
  This explains why the compact path fails and why the full-model fallback is
  needed.
- Add learned-clause database reduction for the flat solver if future large
  instances require many more conflicts.
- Watch for future esawn-like instances with a lower output-group bound; those
  might still benefit from a better direct implicant search. For the current
  `esawn`, the explicit model route appears structurally blocked.

## Known Limitations

- This is still a CDCL-style circuit solver, not a full high-level CBMC trace
  decoder. It may time out or return `UNKNOWN` on the largest instances.
- The proof log can grow with conflicts. Local UNSAT proofs are small enough
  for VeriPB, but very large UNSAT cases would need deletion or stronger proof
  planning.
- The solver is deterministic and does not use comments, filenames, hashes, or
  memorized assignments. Large formulas are handled by structural CNF
  detection and are only reported SAT after checking every original clause.
- The large flat solver is SAT-oriented. If a million-variable instance appears
  UNSAT or outside the supported propagation-heavy CBMC shape, it returns
  `UNKNOWN` rather than attempting an unchecked UNSAT proof.
- Very large SAT instances can emit very large complete models when no compact
  hitting model exists. This is sound because the assignment is checked against
  every original clause before printing, but solver wall time includes the
  model-writing cost.

## Changelog

- Initial scaffold replaced with a C++ watched-literal CDCL solver.
- Added model self-checking before SAT output.
- Added VeriPB RUP proof logging and locally verified UNSAT proofs.
- Added phase saving, polarity schedule, and geometric restarts after an early
  local BMP variant stalled with a single fixed polarity.
- Updated `meta.json` with the recovered CBMC/Tseitin structure and techniques.
- Added a structural guard for million-variable-plus formulas after broker
  runs showed OOM/timeout on the stack-sized instances and invalid SAT model
  checking on the esawn-sized instance.
- Added an optional `LSAT_PHASE_MODE` development override used only for local
  phase-policy timing experiments; the submitted default remains the
  occurrence-polarity schedule.
- Added a conservative topological model/justification attempt based on
  grouping clauses by maximum variable and verifying any produced assignment.
- Added a compact flat-clause CDCL engine for million-variable CBMC formulas.
  It uses linked watch lists and branches first on non-maximum structural input
  variables, solving the two stack formulas and `esawn` locally.
- Changed huge SAT output to complete assignments instead of positive-only
  sparse models, avoiding the earlier model-transport soundness risk.
- Added an 8M-variable SAT-output guard after broker feedback showed the
  locally valid 13.8M-variable `esawn` model is truncated by the verifier's
  64MiB stdout cap and reported `sat-invalid`.
- Replaced the fixed 8M-variable guard with a compact partial-model builder
  that emits a verified true-literal hitting set when it fits under 66.5MB.
- Added a mandatory-literal lower-bound check for compact models. On `esawn`,
  this proves the current full assignment cannot be transported compactly and
  returns `UNKNOWN` quickly.
- Added a guarded partial-assignment early stop for formulas above 10M
  variables. It checks whether the current CDCL trail already hits all original
  clauses and can emit only assigned literals if the result fits the output
  budget.
- Skipped the topological pre-pass on million-variable formulas and added a
  structural all-negative initial phase for formulas above 10M variables,
  reducing local `esawn` solving time to about 13 seconds before the output
  guard.
- Added `tools/check_model.py` for local validation of partial SAT models and
  `tools/cover_analyze.cpp` for assignment coverage and mandatory-literal
  analysis.
- Added `tools/implicant_greedy.cpp` as a standalone failed/prototype direct
  implicant experiment for future work.
- Added a capped small-BMP portfolio: a 5,500-conflict proof-producing CDCL
  probe, followed by a 5,200-conflict flat SAT probe and then the full proof
  fallback. This keeps fast UNSAT proofs intact and cuts the slow BMP SAT
  variants locally.
- Added a structural ordered-CNF branch heuristic for stack-sized flat solving.
  Ordered stack formulas now branch over all variables; shuffled variants keep
  primary-first branching to preserve compact model size.
- Added a no-proof SAT probe for medium low-ratio CBMC formulas. It removes
  unnecessary SAT proof logging on SAT outcomes while preserving the
  proof-producing fallback for UNSAT or unsupported cases.
- Added `tools/implicant_lower_bound.cpp`, which proves the current `esawn`
  instance requires more explicit model literals than the broker stdout window
  can transport.
- Updated `meta.json` and README notes for the small portfolio, ordered stack
  branching, and `esawn` transport lower bound.
- Added a full-model fallback for huge SAT formulas whose compact hitting
  model exceeds the old output-size target. Local `esawn` now returns SAT with
  a 128MB complete model, and the worker's streaming SAT verifier accepts it.
- Replaced the medium vector-clause no-proof SAT probe with the flat-clause
  engine. Local `mix034` dropped from about 15s to about 4.8s, while local
  true/UNSAT medium formulas still use the proof-producing fallback.
