Monday 19 December 2022

Data Deletion On Splunk Indexer To Free Up Space

Here are the ways that you can delete data from your index: Remove indexes and indexed data

The delete command will remove data from an index, but does not recover the disk space.
Using splunk clean to clean an index will remove all data and recover the disk space; there is no way to do this by date.

e.g.: splunk clean eventdata -index <index_name> (In order to clean, Splunkd must not be running.)

It will delete the indexed data permanently. Very very imporant to note that you specify the index name otherwise it will delete all indexes data.

You can set retention time for an index in indexes.conf by using the frozenTimePeriodInSecs setting

[yourindex]
#other index settings
frozenTimePeriodInSecs = 31556926

31556926 is the number of seconds in a year.

You will need to restart Splunk for this setting to take effect. Splunk will immediately began to remove data older than one year from the index.

Sunday 18 December 2022

Splunk Index Directory Structure

Each index occupies a set of directories on the disk. By default, these directories live in $SPLUNK_DB, which, by default, is located in $SPLUNK_HOME/var/lib/splunk.

Look at the following stanza for the main index:

[main] 
homePath = $SPLUNK_DB/defaultdb/db 
coldPath = $SPLUNK_DB/defaultdb/colddb 
thawedPath = $SPLUNK_DB/defaultdb/thaweddb 
maxHotIdleSecs = 86400 
maxHotBuckets = 10 
maxDataSize = auto_high_volume

If our Splunk installation lives at /opt/splunk, the index main is rooted at the path /opt/splunk/var/lib/splunk/defaultdb.

To change your storage location, either modify the value of SPLUNK_DB in $SPLUNK_HOME/etc/splunk-launch.conf or set absolute paths in indexes.conf.

Source - Orielly Splunk