Skip to main content

hackagent.risks.profile_types

Data types for threat-to-evaluation mapping.

Defines the dataclasses that map a vulnerability (threat) to the datasets, attack techniques, objectives, and metrics needed to build an evaluation campaign.

Relevance Objects​

class Relevance(Enum)

How closely a dataset/attack matches a vulnerability.

PRIMARY​

Directly designed to test this vulnerability.

SECONDARY​

Useful for broader coverage or baseline comparison.

DatasetRecommendation Objects​

@dataclass(frozen=True)
class DatasetRecommendation()

Links a dataset preset to a vulnerability with a relevance tag.

Parameters​

preset : str Key in hackagent.datasets.presets.PRESETS (e.g. "advbench"). relevance : Relevance How directly this dataset tests the vulnerability. rationale : str One-liner explaining why this dataset is relevant.

AttackRecommendation Objects​

@dataclass(frozen=True)
class AttackRecommendation()

Links an attack technique to a vulnerability.

Parameters​

technique : str Key in hackagent.attacks.registry.ATTACK_REGISTRY relevance : Relevance How well-suited this technique is for the vulnerability. rationale : str One-liner explaining why this technique applies.

ThreatProfile Objects​

@dataclass(frozen=True)
class ThreatProfile()

Complete evaluation mapping for a single vulnerability.

A ThreatProfile answers the question:

"Given vulnerability *X*, which datasets, attack techniques,
 objective, and metrics should an evaluation campaign use?"

Parameters​

vulnerability : type[BaseVulnerability] The vulnerability class this profile describes. datasets : list[DatasetRecommendation] Recommended datasets, ordered by relevance (primary first). attacks : list[AttackRecommendation] Compatible attack techniques. objective : str Default attack objective key (e.g. "jailbreak", "harmful_behavior", "policy_violation"). metrics : list[str] Metric names relevant to this vulnerability (e.g. "asr", "toxicity_score", "judge_score"). description : str Human-readable summary of what the profile evaluates.

name​

@property
def name() -> str

Vulnerability class name.

primary_datasets​

@property
def primary_datasets() -> List[DatasetRecommendation]

Return only primary-relevance datasets.

secondary_datasets​

@property
def secondary_datasets() -> List[DatasetRecommendation]

Return only secondary-relevance datasets.

primary_attacks​

@property
def primary_attacks() -> List[AttackRecommendation]

Return only primary-relevance attacks.

dataset_presets​

@property
def dataset_presets() -> List[str]

Flat list of all recommended dataset preset keys.

attack_techniques​

@property
def attack_techniques() -> List[str]

Flat list of all recommended attack technique keys.

has_datasets​

@property
def has_datasets() -> bool

True if at least one dataset is recommended.