Overview
To promote reproducibility and openness in privacy, security, and trust research, PST 2026 will run an optional Artifact Evaluation (AE) process for accepted papers. Authors of accepted papers are strongly encouraged to submit their artifacts for evaluation. Artifacts that pass evaluation will receive IEEE reproducibility badges, displayed prominently on the paper's IEEE Xplore page.
Important Dates
- Notification to authors (paper):
June 1, 2026 June 8, 2026
- Artifact Submissions due:
June 10, 2026 June 14, 2026
- Artifact decision notification: June 29, 2026
- Final packaged artifact due: July 3, 2026
IEEE Reproducibility Badges
PST 2026 awards badges in accordance with IEEE's artifact badging policy for code and/or dataset reproducibility. For each of these artifacts, authors choose which badge(s) to target at submission time. The three badge levels are:
- Available: The code and/or datasets, including associated data and documentation, are publicly available in a permanent repository with a persistent identifier (e.g., DOI).
- Reviewed: The code and/or datasets, including associated data and documentation, have been reviewed by the Artifact Evaluation Committee and are reasonable and complete, run to produce the outputs described, and can support reproducibility of the published results.
- Reproducible: An additional step has been taken to certify that an independent party has regenerated computational results using the author-created research objects, methods, code, and conditions of analysis. This badge subsumes Reviewed.
In general, artifacts targeted for the Reviewed or Reproducible badges should be: consistent with the paper, as complete as possible, well documented, and easy to use and re-run.
Process
Artifact evaluation will take place after paper acceptance notifications are sent to authors. To participate, authors must separately submit their artifacts using the
submission link and choose the “PST 2026 Artifact Evaluation” track.
At least two Artifact Evaluation Committee (AEC) members review each submission in depth, installing and running the artifact(s) and evaluating them against the criteria for the requested badge(s). Reviews are
single-blind: authors are known to reviewers, but reviewer identities are not disclosed to authors. Authors must not attempt to de-anonymize reviewers through scripts or trackers embedded in their artifacts or file servers. Attempts at de-anonymization may disqualify submissions from being awarded badges. All reviews are strictly confidential, and best efforts will be made to run each submission on private hardware.
AEC members may contact authors through the submission system during evaluation to resolve issues or ask clarifying questions, while preserving reviewer anonymity. At least one author must be reachable and able to respond to questions in a timely manner throughout the evaluation period. Slow or non-responsive authors risk having their artifact excluded from evaluation.
After evaluation decisions are made, artifacts must be deposited in a repository that guarantees permanent public access. One popular option is
Zenodo, but depending on restrictions imposed by funding agencies, other options are acceptable with the approval of AEC members. Artifacts hosted solely on personal websites or on platforms such as GitHub or GitLab do
not qualify for permanent archival, though authors are welcome to also link to such repositories as supplemental dissemination. A permanent identifier on the storage platform is required and should be included in the camera-ready paper if badges are awarded. Authors who need a delayed public release (e.g., due to an ongoing embargo) should contact the AEC chairs in advance to discuss options.
What to Submit
Authors define the scope of their own artifact submission. Typical artifacts include: source code and build instructions, datasets and raw experimental data, analysis and plotting scripts, survey instruments and results, mechanized proofs, and benchmark suites.
To facilitate evaluation, artifacts submitted for the Reviewed or Reproducible badges should be packaged so that an AEC member can realistically run them. Specifically:
- Package your artifact in a self-contained VirtualBox VM, or Docker container image, or equivalent environment that can be run on commonly available hardware.
- Do not hardcode paths, addresses, or machine-specific identifiers.
- Provide a description of the artifact that explains: what the artifact contains, how to install and run it, how to reproduce each key result or figure in the paper, and the expected outputs. Important: list any special requirements (e.g., storage, GPU, RAM) that are needed.
- For each major result, figure, or table in the paper, include a script or clear instructions for regenerating it.
If your artifact involves malicious or potentially destructive operations by design (e.g., exploits, fuzzing infrastructure), flag this clearly in your submission and contact the AEC chairs before submitting so that appropriate precautions can be arranged.
AEC members will evaluate artifacts on hardware they have access to. If your artifact requires specialized hardware, clearly document those requirements and, where possible, arrange remote access for reviewers. Contact the AEC chairs early if you anticipate this situation.
Reviewer Recruitment
The AEC is looking for volunteers to review PST 2026 artifact submissions. Students and accepted paper authors are encouraged to participate but authors will not be assigned to review their own submissions.
Serving on the AEC is a meaningful form of community service and an opportunity to see first-hand what makes a strong artifact submission — directly useful when preparing your own future submissions. AEC service will be acknowledged on the PST 2026 website and is suitable for inclusion on a CV.
If you are interested in serving as an AEC reviewer, please contact the artifact evaluation co-chairs.
Questions
For questions about the AE process, packaging requirements, or special circumstances, please contact the AEC co-chairs at simone.soderi [AT] imtlucca.it and davidbarrera [AT] cunet.carleton.ca .