a9s MariaDB Cluster Recovery
It occurs that a MariaDB cluster member is not following the master node. In this case, we need to bootstrap this lost member again.
You can achieve this by doing these steps (execute all commands on the lost cluster member):
Replace the <replace-me>
value for the following option below depending on the version:
- For a9s MariaDB >= 10.6 use
mariadb
. - For a9s MariaDB < 10.6 use
mysql
.
Execute
monit stop all
to stop all processesMake sure that there are no
<replace-me>
processes running anymore. You can useps aux
to search for still running processes:ps aux | grep <replace-me>
The PID is in the second column. To kill the process and its child processes, use:
kill -9 {PID}
Example:
kill -9 11496
Remove the directory
/var/vcap/store/<replace-me>*
rm -rf /var/vcap/store/<replace-me>*
Make sure that both
/var/vcap/store/<replace-me>
/var/vcap/store/<replace-me>-temp
are gone:ls /var/vcap/store
- Run the script
/var/vcap/jobs/<replace-me>/bin/pre-start
as root user - Execute
monit start all
to start the processes again.