Security questionnaires and continuous security ratings measure fundamentally different things, and the strongest vendor risk management programs use both rather than choosing one. A questionnaire (SIG, CAIQ, or a custom form) is self-reported, point-in-time evidence of a vendor's internal controls — policies, procedures, and configurations you cannot observe from outside their network. A security rating (BitSight, SecurityScorecard, and similar platforms) is an external, continuously updated scan of what's publicly observable — open ports, certificate hygiene, patch cadence signals, leaked credentials. One tells you what a vendor says they do internally; the other tells you what's actually visible from the outside, updated automatically every day. Neither substitutes for the other.
If you're choosing between them because of budget or time constraints, you're solving the wrong problem — the real design question is how to combine them: which vendors get which treatment, at what cadence, and what triggers escalation from one to the other. That's what the rest of this post works through.
What each approach actually measures
A questionnaire asks a vendor to describe their environment: Do you encrypt data at rest? Do you run background checks on employees with data access? What's your incident response plan? What subprocessors touch our data? None of that is visible from the internet — you're relying on the vendor to answer honestly, and on your team to verify with evidence (SOC 2 report, pen test summary, policy documents) rather than take the checkbox at face value.
A rating platform never asks the vendor anything. It scans what's publicly reachable — the vendor's IP ranges, DNS records, TLS certificates, exposed services, patch-level fingerprints, dark-web credential leaks, botnet/malware chatter tied to their infrastructure — and produces a score, refreshed automatically as conditions change. It requires zero vendor cooperation and updates continuously, which is exactly what a point-in-time questionnaire structurally cannot do.
Comparison: questionnaires vs. security ratings
| Dimension | Security Questionnaires (SIG, CAIQ, custom) | Continuous Security Ratings (BitSight, SecurityScorecard, similar) |
|---|---|---|
| What it measures | Self-reported internal controls, policies, and procedures | Externally observable technical posture and hygiene |
| Timing | Point-in-time snapshot (as of the date answered) | Continuous, typically refreshed daily |
| Vendor cooperation required | Yes — vendor must complete and provide evidence | No — fully passive, outside-in scan |
| Effort/cost per vendor | High: SIG Core averages 5-10 business days; CAIQ 1-2 days for a prepared vendor | Low per vendor once subscribed; flat platform cost covers the whole portfolio |
| Visibility into internal controls | Direct (self-reported, verifiable with evidence requests) | None — cannot see inside the vendor's network |
| False positive risk | Low on facts stated, but self-reported answers can be optimistic or stale | Moderate — IP misattribution, stale subdomains, and shared infrastructure can misfire |
| Scalability across a large vendor portfolio | Poor — labor-intensive per vendor, hard to do for hundreds of vendors | Excellent — one platform monitors the whole portfolio automatically |
| Detects drift between reviews | No — blind between assessment cycles | Yes — this is its core strength |
| Best use case | High-risk, high-access vendors; regulatory/contractual requirement; initial due diligence | Continuous monitoring across the full portfolio; early warning between formal reviews; low-risk vendor triage |
| Which should I use? | Use for onboarding depth and any vendor with real data access — you need to see inside the walls | Use for always-on monitoring and portfolio-wide triage — you need to know the moment something changes outside |
The snapshot problem: why continuous monitoring matters
A questionnaire captures a vendor's posture on the day it was answered. If a vendor passes their SIG Core in January and gets breached through a misconfigured S3 bucket in August, your last formal assessment told you nothing about it — the next scheduled review might not happen for another six to eleven months.
That gap is precisely what ratings close. They can't see the internal control that would have prevented the misconfiguration in the first place — but they can flag the exposed bucket, the newly opened port, or the leaked credential the moment it becomes externally visible, often well before a breach disclosure. Used together, the questionnaire tells you the vendor should be secure based on their stated controls, and the rating tells you whether that's still holding up between now and the next formal review.
Where each approach breaks down
Ratings' most cited weakness is false positives from attribution errors: shared hosting, forgotten subdomains, and partner infrastructure incorrectly scored against the vendor's primary domain. This was a well-documented criticism in the early years of the risk-ratings category, and it dented CISO trust — leadership would present a poor score to a vendor only to learn the flagged asset wasn't even theirs. Rating vendors have since invested in better asset attribution, enrichment, and vendor-facing dispute processes, and platforms like SecurityScorecard now publish studies arguing their scores correlate with breach likelihood. That's a meaningful signal, but it's a correlation across populations — not a guarantee about any single vendor, and it says nothing about internal controls.
Questionnaires' weakness is the mirror image: they only capture what the vendor is willing and able to tell you, and only as of the date answered. A vendor's SIG Core response is a promise about their controls, not proof — which is why mature programs require evidence attachments (SOC 2 Type II, penetration test summaries, insurance certificates) rather than accepting checkbox answers at face value, and why "we passed the questionnaire" and "we verified the controls" are not the same claim.
How to combine them in a real program
A practical, risk-tiered approach:
- Onboarding, all vendors: Run a rating scan immediately (fast, cheap, no vendor effort) to catch obvious external red flags before you invest questionnaire time.
- High-risk / high-access vendors: Full SIG Core or a custom questionnaire at onboarding, evidence-verified, re-assessed annually — plus continuous rating monitoring with alerts on significant score drops.
- Medium-risk vendors: SIG Lite or CAIQ at onboarding, re-assessed every 1-2 years, continuous rating monitoring as the primary signal in between.
- Low-risk vendors: Rating monitoring only after onboarding, with a questionnaire triggered by a material change — new product, new subprocessor, a significant rating drop, or a breach disclosure.
The rating is your tripwire; the questionnaire is your depth check when the tripwire fires or the calendar says it's time. Neither one, run alone, gives you the coverage that combining them does — and given how cheap continuous monitoring is relative to a full questionnaire cycle, there's little reason for a modern vendor risk management program to skip either. For a full breakdown of how to set assessment frequency by vendor risk tier, see our related posts on vendor assessment frequency and how often to reassess vendor security.
Sources: Bitsight vs. SecurityScorecard third-party risk management comparison, CAIQ vs. SIG: what's the difference (Bitsight), SIG questionnaire guide (Vanta), CAIQ vs SIG assessment (HyperComply)