TOPIC:

Assuming the commented out values are the defaults, the ones different that I need to set based on your list above are:

FENCE_HA_ONFAIL=0
FENCE_MIN_HOSTS=2
MONITOR_DELAY=15
XAPI_COUNT=2
XAPI_DELAY=10
XE_TIMEOUT=10

since PROMOTE_SLAVE and SLAVE_HA are both commented out with being set to "1" I assume this is the default. I will need to reboot again to get out of the current hung state, so it'll take me a bit to get back to you again. Much appreciated once again and I'll keep you posted on the next steps! Best regards,
--Tobias

Hi Tobias
We store all settings in a shared pool DB. The config file is there only if you need to override a setting for a specific host in the pool (this is rare).

All settings are centrally managed via the "ha-cfg" command. Changes can be made from any single host in the pool and the change is made live for all pool members within a few seconds.

For ex. To disable logging:

"ha-cfg set enable_logging 0"

This would disable logging for all hosts in the pool.

To view the running config "ha-cfg get"

for a list of CLI options

"ha-cfg --help"

Here's what ha-cfg get now returns (slightly sanitized), so I think I'm ready to test:

DISABLED_VAPPS=()
ENABLE_LOGGING=1
FENCE_ACTION=stop
FENCE_ENABLED=1
FENCE_FILE_LOC=/etc/ha-lizard/fence
FENCE_HA_ONFAIL=0
FENCE_HEURISTICS_IPS=10.15.9.1
FENCE_HOST_FORGET=0
FENCE_IPADDRESS=
FENCE_METHOD=POOL
FENCE_MIN_HOSTS=2
FENCE_PASSWD=
FENCE_QUORUM_REQUIRED=1
FENCE_REBOOT_LONE_HOST=0
FENCE_USE_IP_HEURISTICS=1
GLOBAL_VM_HA=0
MAIL_FROM=noreply@myorganization.xyz
MAIL_ON=1
MAIL_SUBJECT="SYSTEM_ALERT-FROM_HOST:$HOSTNAME"
MAIL_TO=sysadmin@myorganization.xyz
MONITOR_DELAY=15
MONITOR_KILLALL=1
MONITOR_MAX_STARTS=20
MONITOR_SCANRATE=10
OP_MODE=2
PROMOTE_SLAVE=1
SLAVE_HA=1
SLAVE_VM_STAT=0
SMTP_PASS=""
SMTP_PORT="25"
SMTP_SERVER=mailgateway.myorganization.xyz
SMTP_USER=""
XAPI_COUNT=2
XAPI_DELAY=10
XC_FIELD_NAME='ha-lizard-enabled'
XE_TIMEOUT=10
[root@itsxd2 scripts]# ./ha-cfg get
DISABLED_VAPPS=()
ENABLE_LOGGING=1
FENCE_ACTION=stop
FENCE_ENABLED=1
FENCE_FILE_LOC=/etc/ha-lizard/fence
FENCE_HA_ONFAIL=0
FENCE_HEURISTICS_IPS=10.15.9.1
FENCE_HOST_FORGET=0
FENCE_IPADDRESS=
FENCE_METHOD=POOL
FENCE_MIN_HOSTS=2
FENCE_PASSWD=
FENCE_QUORUM_REQUIRED=1
FENCE_REBOOT_LONE_HOST=0
FENCE_USE_IP_HEURISTICS=1
GLOBAL_VM_HA=0
MAIL_FROM=noreply@myorhanization.xyz
MAIL_ON=1
MAIL_SUBJECT="SYSTEM_ALERT-FROM_HOST:$HOSTNAME"
MAIL_TO=me@myorganization.xyz
MONITOR_DELAY=15
MONITOR_KILLALL=1
MONITOR_MAX_STARTS=20
MONITOR_SCANRATE=10
OP_MODE=2
PROMOTE_SLAVE=1
SLAVE_HA=1
SLAVE_VM_STAT=0
SMTP_PASS=""
SMTP_PORT="25"
SMTP_SERVER=mailgateway.myorganization.xyz
SMTP_USER=""
XAPI_COUNT=2
XAPI_DELAY=10
XC_FIELD_NAME='ha-lizard-enabled'
XE_TIMEOUT=10

Still seeing promote slave is disabled. ???


Sep 8 16:33:27 itsxd2 ha-lizard: 18714 update_global_conf_params: Successfully updated global pool configuration settings in /etc/ha-lizard/ha-lizard.pool.conf.
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 update_global_conf_params: DISABLED_VAPPS=() ENABLE_LOGGING=1 FENCE_ACTION=stop FENCE_ENABLED=1 FENCE_FILE_LOC=/etc/ha-lizard/fence FENCE_HA_ONFAIL=0 FENCE_HEURISTICS_IPS=10.15.9.1 FENCE_HOST_FORGET=0 FENCE_IPADDRESS= FENCE_METHOD=POOL FENCE_MIN_HOSTS=2 FENCE_PASSWD= FENCE_QUORUM_REQUIRED=1 FENCE_REBOOT_LONE_HOST=0 FENCE_USE_IP_HEURISTICS=1 GLOBAL_VM_HA=0 MAIL_FROM=noreply@myorganization.xyz MAIL_ON=1 MAIL_SUBJECT="SYSTEM_ALERT-FROM_HOST:$HOSTNAME" MAIL_TO=sysadmin@mygorganization.xyz MONITOR_DELAY=15 MONITOR_KILLALL=1 MONITOR_MAX_STARTS=20 MONITOR_SCANRATE=10 OP_MODE=2 PROMOTE_SLAVE=1 SLAVE_HA=1 SLAVE_VM_STAT=0 SMTP_PASS="" SMTP_PORT="25" SMTP_SERVER=mailgateway.myorganization.xyz SMTP_USER="" XAPI_COUNT=2 XAPI_DELAY=10 XC_FIELD_NAME='ha-lizard-enabled' XE_TIMEOUT=10
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 master_ip: Pool Master IP Address = 10.15.9.17
Sep 8 16:33:27 itsxd2 ha-lizard-NOTICE-/etc/ha-lizard/ha-lizard.sh: /etc/ha-lizard/ha-lizard.sh: line 189: [: =: unary operator expected
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 This slave- itsxd2: not permitted to become master
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 check_xapi: Pool Host 10.15.9.17 xapi status = 1
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 Pool Master is OK - calling function check_ha_enabled - updating local status
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 check_ha_enabled: Checking if ha-lizard is enabled for pool: 7942fc76-66ec-f556-2794-f9f4c109d6b6
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 check_ha_enabled: ha-lizard is enabled
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 Checking state file for status if ha-lizard is enabled
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 Statefile /etc/ha-lizard/state/ha_lizard_enabled found: checking if ha-lizard is enabled
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 ha-lizard is enabled - continuing
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 check_xs_ha: Checking XenServer HA status
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 Calling Function write_pool_state - updating local state files
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 get_vms_on_host: Returned 1255d22c-e83f-a326-1297-55932d0b2f67
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 get_vms_on_host: Returned 0cc49275-cb62-d2d1-c5a7-5d5049776ea4
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 check_ha_enabled: Checking if ha-lizard is enabled for pool: 7942fc76-66ec-f556-2794-f9f4c109d6b6
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 check_ha_enabled: ha-lizard is enabled
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 get_pool_host_list: enabled flag set - returning only hosts with enabled=true
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 get_pool_host_list: returned b5773e37-0e67-4055-8857-1c9c31f2dfdc 23177c12-4300-43fa-b5cb-b6f7529c2ffa
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 get_pool_ip_list: returned 10.15.9.17
Sep 8 16:33:27 itsxd2 ha-lizard: 18714 get_pool_ip_list: returned 10.15.9.17 10.15.9.25

I think line 189 in ha-lizard.sh should read:

if [ $THIS_SLAVE_UUID == $AUTOPROMOTE_UUID ]

It looks like either autopromote_uuid failed to set or the slave failed to get its own uuid.