CWE-1434: Insecure Setting of Generative AI/ML Model Inference Parameters

BaseDraft

The product has a component that relies on a generative AI/ML model configured with inference parameters that produce an unacceptably high rate of erroneous or unexpected outputs.

View on MITRE
Back to CWE Lookup

Extended Description

Generative AI/ML models, such as those used for text generation, image synthesis, and other creative tasks, rely on inference parameters that control model behavior, such as temperature, Top P, and Top K. These parameters affect the model's internal decision-making processes, learning rate, and probability distributions. Incorrect settings can lead to unusual behavior such as text "hallucinations," unrealistic images, or failure to converge during training. The impact of such misconfigurations can compromise the integrity of the application. If the results are used in security-critical operations or decisions, then this could violate the intended security policy, i.e., introduce a vulnerability.

Technical Details

Structure
Simple

Applicable To

Languages
Not Language-Specific
Platforms

Learn More

CWE-1434: Insecure Setting of Generative AI/ML Model Inference Parameters | CWE Lookup | Inventive HQ