Moving indexes in Splunk can be required in migrations, upgrades or simply as an exercise for backup and restores.
In this example, we will move an index called SSG from a Splunk 7.02 server called UBUNTU07 running on Ubuntu Linux to another 7.02 Splunk server running on another Ubuntu server called SPLUNKOSAUR. We are assuming that Splunk is installed in the default Linux location (/opt/splunk) and that the indexes are in the default Splunk location (/opt/splunk/var/lib/splunk) – if not, adjust the paths accordingly. For Linux/Windows combinations, the copy commands have to be adjusted accordingly but otherwise the overall process remains the same.
Step 1
On the new server, create an index with the same name. In this case we will create an index called SSG on SPLUNKOSAUR:
Step 2
Stop Splunk on the new server (SPLUNKOSAUR). From a terminal run:
sudo /opt/splunk/bin/splunk stop
Step 3
Remove the folder for the newly created index:
sudo rm -r /opt/splunk/var/lib/splunk/ssg
Step 4
On the old server, remove or disable any inputs that send data to the index that has to be migrated. In our case, we have a file monitor input that ingests a Juniper SSG firewall syslog logs into the SSG index:
Step 5
Stop Splunk on the old server (UBUNTU07):
sudo /opt/splunk/bin/splunk stop
Step 6
Transfer the folder containing the index to be migrated to the new server. There are several ways to transfer folders between servers. In our case, we will use the scp utility (secure copy) that ships with any Ubuntu installation. scp is using SSH so the servers have to run the SSH daemon. If SSH is not installed, the installation is rather simple. In a terminal prompt run:
sudo apt-get install openssh-server
To check the status of the SSH server run:
sudo service ssh status
Once SSH is installed on the receiving server, you can use the scp utility. To transfer a full folder use the following command:
sudo scp -r /opt/splunk/var/lib/splunk/ssg user_name@server_name_or_ip:”/opt/splunk/var/lib/splunk/ssg”
Replace user_name and server_name_or_ip with the corresponding values. In our case, SPLUNKOSAUR has an IP address of 192.168.5.25 and we will use a user called jdoe so the syntax would be:
sudo scp -r /opt/splunk/var/lib/splunk/ssg jdoe@192.168.5.25:”/tmp”
This will transfer the ssg folder to the /tmp folder on the new server.
The user will be prompted for the password and if entered correctly, the copying will be initiated. The duration of the copying process will depend on the amount of data to be transferred and the type of connectivity between servers. If the volume of data is very large and/or the connectivity between servers is slow or inexistent, then the files can be transferred with a portable media such as an USB drive. In any case, the end results have to be the same: the existing SSG index folder from the old server has to end up in the indexes folder on the new server.
Step 7
On the new server, transfer the ssg folder from /tmp to /opt/splunk/var/lib/splunk. Depending on the user that you are logged in as, you may have to become root (i.e. use the “sudo -i” command to facilitate all the rest of the commands that you need to configure the new folder properly):
mv ssg /opt/splunk/var/lib/splunk
After the transfer, the /opt/splunk/var/lib/splunk folder on the new server should look something like this:
Step 8
Note that the ssg folder has as owner and group the user that was used to transfer the folder (jdoe). Adjust the ownership of the SSG folder to match the existing indexes:
chown root:root /opt/splunk/var/lib/splunk/ssg
The folder should indicate root as owner and group:
Step 9
Start Splunk on the new server (SPLUNKOSAUR):
/opt/splunk/bin/splunk start
Login into the new server and verify that the SSG index has all the data from the old server:
Perform a search to validate that the data is available:
Recreate the inputs as needed. This depends on the nature of the previous inputs, i.e. if the input was for local files, the source of the data will have to be transferred to the new server or the old server configured as a forwarder.
Step 10
As the last step, restart the old server and remove the old index.
NOTE: The information here is copied from below link -
