2026-07-11

SVA generation benchmarks for LLMs

Assertion Generation Benchmarks

Title	Authors	Affiliation	Venue
AssertLLM	Fang et al.	HKUST	ICLAD ‘24, ASP-DAC ‘25
FVEval	Kang et al.	UCB, Nvidia	DATE ‘25
AssertionBench	Pulavarthi et al.	UIC, Microsoft	NAACL ‘25
OpenLLM-RTL	Liu et al.	HKUST	ICCAD ‘25
CVDP	Pinckney et al.	Nvidia	arXiv 06/25
AssertLLM2	Wu et al.	HKUST	arXiv 05/26
FIXME	Wan et al.	Southeast University, NCTIEDA, Texas Tech, City University of Hong Kong, Chinese University of Hong Kong	AAAI-26
HierSVA	Nie et al.	University of Washington	arXiv

Title	Approach	Dataset
AssertLLM	Spec	Opencores
FVEval	Prompt+RTL, RTL	Synthetic FSM and Arithmetic Pipeline
AssertionBench	RTL	Opencores
OpenLLM-RTL	Spec	Opencores
CVDP	Prompt+RTL	Nvidia Engineers
AssertLLM2	Spec	Opencores
FIXME	Spec+RTL	Custom
HierSVA	RTL+Design Information	BaseJump STL

FVEval synthetic dataset is derived from just 2 templates - FSM and Arithmetic pipeline. The randomly mutate operators, depth, etc. to create new designs. I don’t think this captures every aspect of digital design. An arithmetic pipeline, however deep it might be, has some innate properties. So we are not really exploring much by just changing the depth.
AssertionBench’s in-context-examples confuses the models. They could have also used more of Opencores. They don’t use the larger designs.
FIXME is closed source.

I cannot comment on HierSVA yet. The other works just use spec documents. This isn’t inherently a red flag. Just a different approach.

I have identified two distinct groups that have published the most in this space - HKUST and Nvidia

Some other HKUST works that I have read so far are liuRTLCoderFullyOpenSource2025 luRTLLMOpenSourceBenchmark2023
The last author Zhiyao Xie seems to be the common link between them.
His github is also where almost all of these works resides