How to Extract IOCs from Text?

The extraction pipeline

Every IOC extractor — whether it is fifteen lines of Python or an enterprise SIEM parser — runs the same five stages. The order matters: refang before you dedupe, because hxxp://evil[.]com and http://evil.com are the same indicator and should collapse into one row.

Extract loosely (steps 1-3), then normalize and validate strictly before you act.

The single most common mistake is skipping stage 3. Threat reports almost never contain live indicators — publishers defang them on purpose so a reader cannot fat-finger a click onto a live command-and-control server. A regex that only matches http:// and literal dots will silently miss the majority of the indicators in a well-written report.

IOC types, patterns, and defanged forms

The table below is the working core of any extractor: each indicator type, a regex that matches it, and the defanged forms you must refang before or during the match. Treat the patterns as extraction patterns — deliberately permissive to maximize recall. Tightening them into valid indicators is the job of the validation stage.

IOC type	Extraction regex (illustrative)	Common defanged forms to refang
IPv4	\b(?:\d{1,3}\.){3}\d{1,3}\b	8[.]8[.]8[.]8, 8(.)8(.)8(.)8, 8[dot]8[dot]8[dot]8
IPv6	\b(?:[A-Fa-f0-9]{1,4}:){2,7}[A-Fa-f0-9]{1,4}\b	colons occasionally wrapped as [:]
Domain	\b(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z]{2,}\b	evil[.]com, evil(.)com, evil[dot]com
URL	`\b(?:https?	hxxps?
Email	\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b	user[@]evil[.]com, user AT evil DOT com
MD5	\b[a-fA-F0-9]{32}\b	rarely defanged; watch git short-hash collisions
SHA1	\b[a-fA-F0-9]{40}\b	rarely defanged; overlaps git commit hashes
SHA256	\b[a-fA-F0-9]{64}\b	rarely defanged
CVE	\bCVE-\d{4}-\d{4,7}\b	occasionally CVE‑ with a non-ASCII hyphen
Windows path	`\b[A-Za-z]:\(?:[^\/:*?"<>	\r\n]+\?)+`
Registry key	`\bHK(?:LM	CU

A few things the patterns above deliberately do not do, because they belong to validation rather than extraction:

• Range-check IPv4 octets. \b(?:\d{1,3}\.){3}\d{1,3}\b will match 999.1.1.1. Extract it; let validation drop it.
• Distinguish MD5 from a truncated SHA. Anchor on \b word boundaries so a 64-character SHA256 is not chopped into a bogus 32-character MD5.
• Verify the TLD is real. A domain regex matches thing.local and file.exe; a validation pass compares the top-level domain against the IANA list.

Refanging: the step everyone forgets

Refanging is the inverse of URL defanging, and it is where a naive grep pipeline falls apart. There is no single standard for how analysts defang, so an extractor has to reverse a family of conventions. The reliable substitutions:

• hxxp → http, hxxps → https, fxp → ftp
• [.], (.), {.}, [dot], (dot), dot → .
• [:], [://] → : / ://
• [@], (at), at → @
• Leading/trailing angle brackets and backticks stripped from URLs

The Python library iocextract handles all of these for you and exposes a refang=True flag so extraction and refanging happen in one call. If you are scripting it, the pattern is short:

import iocextract

with open("report.txt") as f:
    text = f.read()

urls   = list(iocextract.extract_urls(text, refang=True))
ips    = list(iocextract.extract_ips(text, refang=True))
hashes = list(iocextract.extract_hashes(text))
emails = list(iocextract.extract_emails(text, refang=True))

Choosing an extraction tool

You do not need to reinvent the regex table above — several mature tools already encode it, including the defanged forms.

Tool	Form	Best for
Browser IOC extractor	Paste-and-go web tool	Ad-hoc jobs, no install, quick triage of a single report
iocextract	Python library + CLI (pip install iocextract)	Scripting, pipelines, embedding in Python tooling
Cacador	Single Go binary, reads stdin, emits JSON	Fast command-line one-liners, runtimes without Python
SIEM parsers (Splunk, Elastic)	Ingest-time field extraction	Continuous extraction from live log streams at scale
Custom regex + grep	Your own scripts	Full control, learning, one-off spot checks

For a report you have pasted into a browser, the fastest route is a web extractor that runs entirely client-side — no upload, no round trip.

When to reach for each: use a browser tool for a one-off paste; reach for iocextract when the job is part of a Python workflow or you want to chain it into enrichment; pick Cacador for a shell one-liner (cat report.txt | cacador) or a container without a Python runtime; and lean on your SIEM's field extraction when the source is a continuous log stream rather than a static document.

Extract loosely, validate strictly

The guiding principle across every stage above: extraction should maximize recall and validation should maximize precision. Extract with permissive patterns so no indicator hides behind a defanged dot or an unusual encoding, refang everything, then hand the raw candidate list to a strict validation pass that range-checks octets, confirms hash lengths, checks TLDs against IANA, and filters your internal allowlist. Collapsing those two stages into one over-tight regex is how real indicators get silently dropped.

Handling special cases

Real-world text throws edge cases at every extractor:

• Split indicators. URLs wrapped across a line break or a domain broken by a stray space. Some extractors reassemble these; grep will not.
• False positives by shape. Version numbers look like IPv4, git commit hashes look like SHA1, CDN and telemetry domains look malicious, and standard Windows paths look like malware artifacts. Extraction over-collects by design — filtering happens later.
• Mixed encodings. UTF-8 versus ASCII, and non-ASCII look-alike characters (a Cyrillic а inside a domain, or a Unicode hyphen inside a CVE ID). Normalize encoding before matching.
• Deduplication with case. EVIL.COM and evil.com are one domain. Lowercase domains and emails during normalization so dedupe actually collapses them.

Integrating extraction into your workflow

Extracted indicators only create value once they flow into operations. Export the deduplicated set as STIX or CSV into your threat intelligence platform, push high-confidence indicators into SIEM watchlists or firewall blocklists, and keep a note of each indicator's source document so you can retrace provenance during an investigation. Establish a retention and review cycle — stale indicators clutter detection logic and drive false positives as infrastructure gets reused by unrelated parties.

Conclusion

Extracting IOCs from text is a five-stage pipeline — paste, regex-parse per type, refang, normalize and dedupe, export — and the stages that separate a usable indicator list from a noisy one are refanging and the extract-loose-then-validate-strict discipline. Whether you paste a report into a browser extractor, script iocextract into a Python pipeline, or pipe text through Cacador, the mechanics are the same. Get the candidates out completely first; then move to validating what you extracted to decide what is actually worth acting on.