SPLUNK
Regular expresions perl
Create searches.
Ad hoc: http://docs.splunk.com/Documentation/Splunk/5.0/Search/Aboutcustomsearchcommands
http://docs.splunk.com/Documentation/Splunk/6.0.1/Search/Whatsinthismanual
Commands:
regex:
convert
eval
extract (kv)
fields
lookup (if not local=t)
mvexpand
multikv
rename
regex : Removes or keeps results that match the specified regular expression(streamingly). …
| regex _raw="(?=!\d)10\.\d{1,3}\.\d{1,3}\.\d{1,3}(?!\d)" (this is from the field _raw)
replace
rex : string replacement and character substitution.
search
strcat
tagstyper.
Examples: Creates a new field called 'test' taking as key “Error adding limit institutionId:”. Then it deduplicates all the members of the field and sorts them up
source="ParFX-PRD-Env" host="l52prdsrv-mds1.pure.local" error | rex field=_raw "Error adding limit institutionId: (?<test>.*)" | dedup test | sort test
EXTRACT, ORDER & TABULATE FIELDS
To extract fields that are not automatically detected:
- Search. Select Source of interest, for example:
source="network"
- Make a sample search contained log entries with the fields we want to extract
- Click on the small square box with an arrow which are in each the log entries and select extract fields
- Fill up the text box “Example values for a field:” with some examples of the field you want to extract and click “Generate”
- We can click in test, this will generate a simple table with the count and percentage of that field.
index=main host="l52eatfwl-edge1" PROD-DMS created | rex field=_raw "session created (?P<test>\\d+\\.\\d+\\.\\d+\\.\\d+)(?=/)" | dedup test | sort test | table test