Order of stopping/starting Splunk services in a clustered environment

When stopping/starting the Splunk server for the purpose of periodic power outages or server maintenance, not for version upgrades, etc., stop/start the Splunk services in the following order.

[Stop]
① Stop forwarder
②Search head stop
(3) Stop the deployment server
④Stop deployer
(5) Indexer stop
(6) Stop the cluster master
⑦ Suspension of license master

[Start-up]
① Start license master
(2) Start the cluster master
(3) Start indexer
④ Start search head
⑤ Start forwarder
(6) Start the deployment server
⑦ Launch deployer

Specifically, if you want to stop and start all Splunk services, please follow the steps below. Please replace $SPLUNK_HOME in the following steps with the Splunk installation directory.

*For default installation

<Linux>

$SPLUNK_HOME : /opt/splunk

<Windows>

$SPLUNK_HOME : C:\Program Files\splunk

▼Full stop

• Forwarder's Splunk service stop

$SPLUNK_HOME/bin/splunk stop

• Search head Splunk service stop

$SPLUNK_HOME/bin/splunk stop

• Stop the Splunk service on the deployment server

$SPLUNK_HOME/bin/splunk stop

• Stop the Splunk service on the deployer

$SPLUNK_HOME/bin/splunk stop

• Switching to maintenance mode (performed on the cluster master)

$SPLUNK_HOME/bin/splunk enable maintenance-mode --answer-yes -auth admin:<password>

By switching to maintenance mode, you can prevent unnecessary replication processing without judging that the indexer is down when it is stopped. For more information on maintenance mode, please refer to the document below.

http://docs.splunk.com/Documentation/Splunk/6.6.2/Indexer/Usemaintenancemode

• By switching to maintenance mode, you can prevent unnecessary replication processing without judging that the indexer is down when it is stopped. For more information on maintenance mode, please refer to the document below.

https://docs.splunk.com/Documentation/Splunk/9.1.2/Indexer/Usemaintenancemode

• Confirm that it has been switched to maintenance mode (performed on the cluster master)

$SPLUNK_HOME/bin/splunk show maintenance-mode -auth admin:<password>

* When switching to maintenance mode, the output is as follows.

Maintenance mode is : 1

• Adjust timeout value (performed on cluster master)

$SPLUNK_HOME/bin/splunk edit cluster-config
-restart_timeout <秒数> -auth admin:<password>

*Please specify the time it takes to stop the Splunk service for all indexers.

*If it's not too short, a rough time is fine.

*If you set it short, there is a possibility that unnecessary bucket copies will occur and it may take time to start up.

*This timeout value is effective only when stopping with the offline command, so there is no particular need to restore it.

• Stop the Splunk service on the indexer

$SPLUNK_HOME/bin/splunk offline

• Stop Splunk service on cluster master

$SPLUNK_HOME/bin/splunk stop

• Stop Splunk service on license master

$SPLUNK_HOME/bin/splunk stop

▼ Full start

• Start the Splunk service on the license master

$SPLUNK_HOME/bin/splunk start

• Start Splunk service on cluster master

$SPLUNK_HOME/bin/splunk start

• Switching to maintenance mode (performed on the cluster master)

$SPLUNK_HOME/bin/splunk enable maintenance-mode --answer-yes -auth admin:<password>

*In Splunk 6.5.x and earlier versions, maintenance mode is disabled after restarting, so it is necessary to switch to maintenance mode again.

• Confirm that it has been switched to maintenance mode (performed on the cluster master)

$SPLUNK_HOME/bin/splunk show maintenance-mode -auth admin:<password>

* When switching to maintenance mode, the output is as follows.

Maintenance mode is : 1

• Start indexer Splunk service

$SPLUNK_HOME/bin/splunk start

• Disable maintenance mode (done on cluster master)

$SPLUNK_HOME/bin/splunk disable maintenance-mode -auth admin:<password>

• Make sure maintenance mode is disabled (done on cluster master)

$SPLUNK_HOME/bin/splunk show maintenance-mode -auth admin:<password>

*When the maintenance mode is disabled, the output is as follows.

Maintenance mode is : 0

• Start Splunk service for search head

$SPLUNK_HOME/bin/splunk start

• Start Splunk service on forwarder

$SPLUNK_HOME/bin/splunk start

• Start the Splunk service on the deployment server

$SPLUNK_HOME/bin/splunk start

• Start the deployer's Splunk service

$SPLUNK_HOME/bin/splunk start