Grok Pattern Builder & Debugger

Build and debug a Grok pattern against your own log lines

Writing a Grok pattern is a tight loop: change one token, see whether the line still parses, change it back. Doing that loop inside Logstash means restarting a pipeline for every attempt. This tool runs the loop in the browser — paste real log lines, type a pattern, and the matched fields, the failure position and the fix suggestions all update as you type. Nothing is uploaded; the pattern is compiled to a JavaScript regular expression and executed on the page, so you can paste production log excerpts without them leaving your machine.

When the pattern is right, the output panel hands you the same expression as a Logstash filter block, an Elasticsearch or OpenSearch ingest pipeline, or a plain regular expression in JavaScript or PCRE form.

The syntax, in four lines

Everything outside a %{…} token is literal — and literal here means regex, not plain text. That single fact causes most first-attempt failures. A log line containing [ERROR] needs \[%{LOGLEVEL:level}\] in the pattern, because unescaped square brackets are a character class. The same applies to ( ) { } . * + ? | ^ $. When a literal fails to match, the debugger checks specifically for unescaped brackets and parentheses and offers the escaped version as a one-click fix.

The type suffix is worth using consistently. %{NUMBER:status} stores 200 as the string “200”; %{NUMBER:status:int} stores it as a number you can aggregate, range-query and graph. The tool applies the same coercion when it displays extracted fields, so a value shown without quotes in the results table is one your log platform will index as numeric.

Start from a log line, not from a blank pattern

Two shortcuts get you most of the way before you type a token.

The preset selector loads a pattern and a matching sample log together. There are twelve, covering the formats people actually parse: nginx access, Apache combined (using the built-in %{COMBINEDAPACHELOG}), BSD syslog (RFC 3164), SSH authentication failures from auth.log, Cisco ASA connection messages, a generic ISO-8601 application log, Java Log4j/Logback lines, AWS ELB access logs, HAProxy HTTP logs, PostgreSQL server logs, IIS W3C extended logs, and the Docker/containerd line prefix. Loading a preset and then editing it against your own line is usually faster than starting clean, because the hard part — the timestamp — is already correct.

The suggest button takes the first non-empty line of your log box and generates a pattern for it, walking the line left to right and emitting a token wherever it recognises a known structure. It tries the most specific recognisers first — ISO-8601 timestamps, Apache dates, syslog timestamps, URIs, email addresses, UUIDs, MACs, IP addresses, quoted strings, log levels, paths, then numbers and words — and escapes everything else as a literal. If the tail of the result degenerates into a long run of %{WORD} and %{NUMBER} tokens, it collapses that tail into %{GREEDYDATA:message}. Treat the output as a draft: it names fields generically (word2, num3) and it will happily match a value that is only coincidentally shaped like a number.

The pattern library

Seventy-two named patterns ship with the tool, taken from the standard logstash-patterns-core dictionary and adapted to compile under a JavaScript regex engine. They are grouped and searchable in the side panel, and clicking one appends it to your pattern. The distribution tells you where the complexity lives:

Patterns compose: %{IP} is defined as %{IPV6}|%{IPV4}, and %{COMBINEDAPACHELOG} is %{COMMONAPACHELOG} plus a referrer and user agent. The compiler expands these references recursively, so using a high-level pattern costs you nothing and reads far better than the expansion.

A worked example

The tool opens with this pattern:

%{TIMESTAMP_ISO8601:timestamp} \[%{LOGLEVEL:level}\] %{NOTSPACE:logger} - %{GREEDYDATA:message}

and this line:

2026-01-15T10:30:45.123Z [ERROR] com.example.PaymentService - Failed to process payment for order 12345: connection timeout

Reading it token by token: TIMESTAMP_ISO8601 takes the leading timestamp including the fractional seconds and the Z; the escaped brackets consume the literal [ and ] around ERROR, which LOGLEVEL matches (it accepts ERROR, ERR, Error and the rest of the usual spellings, in several cases); NOTSPACE takes the dotted logger name because it runs to the next space; the literal  -  separates; and GREEDYDATA takes the remainder of the line. Four fields, and the results table shows each captured value with its position highlighted in the source line.

Now the useful part: break it. Change %{LOGLEVEL:level} to %{WORD:level} and it still matches — WORD is looser but wide enough. Change it to %{NUMBER:level} and the line fails; the debugger reports that %{NUMBER} does not match ERROR] com.example… at that position, names the patterns that would match there, and offers each as a one-click replacement.

The GREEDYDATA trap

%{GREEDYDATA} is .* and %{DATA} is .*? — greedy and lazy. That difference is the single most common cause of a pattern that “works” but captures the wrong text.

Put GREEDYDATA anywhere but the end and it first swallows the entire line, then backtracks only as far as it must to let the rest of the pattern match. Because it gives back as little as possible, every field after it gets the last candidate on the line rather than the first. A pattern like %{GREEDYDATA:prefix} %{NUMBER:id} against a line containing several numbers captures the final one, silently, with no error to tell you. Two consequences follow:

• Use %{DATA} between known anchors, and %{GREEDYDATA} only at the end. Lazy matching between two literals you can actually see in the line does what you meant.
• Expect backtracking cost. Every greedy token in the middle multiplies the work the engine does on a line that ultimately fails to match, and lines that fail are exactly the ones a busy pipeline sees most.

The other side of the same coin: a Grok pattern is not anchored. If your pattern matches starting at character 40, the tool tells you so and says how many leading characters were skipped, because an unanchored pattern that quietly matches part of a line is worse than one that fails loudly. It also tells you when a match leaves trailing characters uncaptured, and offers to append %{GREEDYDATA:rest} — the correct trailing use.

The debugger’s failure analysis

When a line does not match, the tool does not just say “no match”. It recompiles your pattern one segment at a time and finds the longest prefix that still matches, which pinpoints the exact character where parsing stopped. From there it produces targeted suggestions:

• Whitespace mismatch. A single literal space in the pattern against a tab or a run of spaces in the log — overwhelmingly the most common failure. The suggested fix replaces the space with \s+.
• Wrong pattern for the text. It tests the remaining text against the specific-first recogniser list and tells you which pattern does match there, preserving your field name and type suffix in the replacement.
• No pattern matches. It falls back to suggesting %{DATA:value} (lazy) or %{NOTSPACE:value} (to the next space).
• Unknown pattern name. A misspelled %{TIMSTAMP_ISO8601} gets a “did you mean” list computed by edit distance against the dictionary, each with a one-click fix.
• Literal mismatch. It shows what it expected versus what the line actually contains at that offset.

The panel analyses the first few problem lines rather than all of them, which is the right trade: log lines that fail usually fail the same way, and fixing the first fixes the rest.

Custom patterns

The custom patterns box takes the same format as a Logstash patterns file — one definition per line, name then a space then the regex source, with # for comments:

ORDERID ORD-\d{6}

Definitions may reference other patterns with %{…}, and a custom name overrides a built-in of the same name. When you export, custom definitions travel with the pattern: the Logstash output emits them as a pattern_definitions block and the ingest-pipeline output as a pattern_definitions object inside the grok processor, so what you copy is complete.

Exports, and the one caveat that matters

Five output formats are available: the raw Grok pattern; a JavaScript regular expression; a PCRE-style expression with named groups for Python, grep -P or Go; a complete Logstash filter { grok { … } } block with quoting and backslashes escaped for the config file; and an Elasticsearch/OpenSearch ingest pipeline as JSON. The pattern itself is also shareable — pattern and log are encoded into the page URL (the log only up to a couple of thousand characters), so a link reproduces the exact state you were debugging.

The caveat: this tool compiles to JavaScript’s regex engine, and Logstash uses Oniguruma. The dictionary here has atomic groups rewritten as ordinary non-capturing groups to compile at all. Matching semantics are the same for every pattern in normal use — what differs is backtracking behaviour on pathological input, so a pattern that is merely slow here can be slower in production, and a catastrophic-backtracking risk is not something this tool will surface for you. Validate the fields, then load-test the pipeline separately. The other structural difference is that Grok in Logstash tries a list of patterns and applies further filters afterwards; here you test one pattern against many lines, which is the right granularity for getting a single pattern correct but is not a substitute for testing the whole pipeline.