Forum
Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1

TOPIC: drbd status - diskless

drbd status - diskless 1 month 2 weeks ago #1961

  • gerry kernan
  • gerry kernan's Avatar Topic Author
  • Offline
  • Posts: 4
hi

just noticed that one of our 2 node setups has stopped replication via DRBD . primary is showing as diskless .
is there a procedure to recover from this . i'm guess that the secondary node will be out of sync so i cant fail-over to that to run the VM's

[root@xensrva ~]# cat /proc/drbd
version: 8.4.5 (api:1/proto:86-101)
srcversion: 2A6B2FA4F0703B49CA9C727

1: cs:Connected ro:Primary/Secondary ds:Diskless/UpToDate C r
ns:0 nr:8505572 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

Please Log in or Create an account to join the conversation.

drbd status - diskless 1 month 2 weeks ago #1963

  • Salvatore Costantino
  • Salvatore Costantino's Avatar
  • Offline
  • Posts: 613
if no changes (maintenance, reboots, etc) have been made and the block device shows as diskless, it may be a failing disk.

Chances are your VMs are not working either. If that is the case, the secondary would have your most up to date data, but, do not take my word on that. You will need to confirm this before invalidating any data.

If this is the case, you can enter manual mode and promote the slave to the primary mode, however, this could fail depending on which disk is seen as most up to date by drbd.

If you are certain that your disk is OK, you can try to detach and reattach the disk from the drbdadm utility.

you could also try re-initiating all services on the master.
service iscsi-ha-watchdog stop
service iscsi-ha stop
service drbd stop
service tgtd stop

once everything is stopped, you can re-start the storage controller daemon which will ensure that everything comes up in the correct order.
service iscsi-ha start

Please Log in or Create an account to join the conversation.

drbd status - diskless 1 month 2 weeks ago #1964

  • gerry kernan
  • gerry kernan's Avatar Topic Author
  • Offline
  • Posts: 4
Hi Salvatore
thanks for getting back to me so quick .
think i found the problem . had lots of these entries in user.log about duplicate PV

Jan 8 11:52:03 xensrva iscsi-ha: 11977 check_drbd_resource_state: DRBD Resource: iscsi1 in Primary mode
Jan 8 11:52:03 xensrva iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: Found duplicate PV eCCVC9K8ncE0V3HlYONKdikdtnYUXRUE: using /dev/drbd1 not /dev/sdb
Jan 8 11:52:03 xensrva iscsi-ha-ERROR-/etc/iscsi-ha/init/iscsi-ha.mon: Using duplicate PV /dev/drbd1 from subsystem DRBD, ignoring /dev/sdb
Jan 8 11:52:13 xensrva iscsi-ha: 12198 validate_drbd_resources_loaded: Checking DRBD has loaded with resources. Checking [ 5 ] > [ 2 ]


i found that i had mistake in lvm.conf i had
filter = [ "r|/dev/sdb|", "a|sd.*|", "r|/dev/VG_Xen.*/*|", "r|/dev/drbd.*|", "r|/dev/VGX.*|"]

would this be causing the issue.
ive corrected it to
filter = [ "r|/dev/sdb|", "r|sd.*|", "r|/dev/VG_Xen.*/*|", "r|/dev/drbd.*|", "r|/dev/VGX.*|"]

Gerry

Please Log in or Create an account to join the conversation.

drbd status - diskless 1 month 2 weeks ago #1965

  • Salvatore Costantino
  • Salvatore Costantino's Avatar
  • Offline
  • Posts: 613
Hi Gerry,
Yes, an incorrect filter would cause LVM to read signatures from both the DRBD device and the underlying block device, which are the same.

FYI - you will also need to update the filter in /etc/lvm/master/lvm.conf to match

Please Log in or Create an account to join the conversation.

  • Page:
  • 1