Forum
Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1

TOPIC:

drbd continuously stalling and slow on initial sync 3 years 4 months ago #2250

  • Christoph Christ
  • Christoph Christ's Avatar Topic Author
  • Offline
  • Posts: 1
version: 8.4.10 (api:1/proto:86-101)
srcversion: CE41F2FAB70AE3534BF40A6

1: cs:SyncTarget ro:Primary/Secondary ds:Inconsistent/UpToDate C r---n-
ns:726530524 nr:963289216 dw:1689893348 dr:49414172 al:180408 bm:0 lo:0 pe:18969 ua:0 ap:18969 ep:1 wo:f oos:10212518276
[>...................] sync'ed: 8.5% (9973160/10899440)M
finish: 4498:49:12 speed: 628 (3,832) want: 102,400 K/sec (stalled)

a few minutes later on:

version: 8.4.10 (api:1/proto:86-101)
srcversion: CE41F2FAB70AE3534BF40A6

1: cs:SyncTarget ro:Primary/Secondary ds:Inconsistent/UpToDate C r---n-
ns:727402076 nr:964469212 dw:1691980128 dr:49788344 al:180531 bm:0 lo:0 pe:27739 ua:0 ap:27739 ep:1 wo:f oos:10211338280
[>...................] sync'ed: 8.6% (9972008/10899440)M
finish: 2644:34:31 speed: 1,068 (3,832) want: 102,400 K/sec

drbd.conf:

global { usage-count no; }
common { syncer { rate 100M; } }
resource iscsi1 {
protocol C;


net {
after-sb-0pri discard-zero-changes;
after-sb-1pri consensus;
cram-hmac-alg sha1;
shared-secret PUTyourSECREThere;
# max-epoch-size 20000;
max-buffers 120000;
#sndbuf-size 1024k ;
#rcvbuf-size 2048k;

}
disk {
c-plan-ahead 0;
c-fill-target 24M;
c-min-rate 80M;
c-max-rate 120M;
}

These 'Servers have gigabit network cards, everything in default halizard installer setup on two freshly installed xcp-ng version 8.1, also default setup. Any idea how to improve speed?

Kind regards
Christoph

Please Log in or Create an account to join the conversation.

Last edit: by Christoph Christ.

drbd continuously stalling and slow on initial sync 3 years 4 months ago #2253

  • Salvatore Costantino
  • Salvatore Costantino's Avatar
  • Offline
  • Posts: 722
Looks like you have a large number of out of sequence blocks which only decreased marginally in the time between your 2 tests.

Some things to look for:
1- potential network layer issue. Check for layer 2 issues on the interfaces used for DRBD
2- Any chance your DRBD connection is not dedicated? Meaning is it shared with management and VM traffic? For production, this should really be a dedicated link between your hosts.
3- Are your DRBD interfaces connected to a switch or are the interfaces directly attached to eachother?
4- During initial sync, it is possible to create and launch new VMs. This would adversely affect the initial sync speed and should eventually catch up, assuming the load (writes) from any running VM is not too much. If you have any running VMs on top of the DRBD storage, try shutting them down if you can.

Please Log in or Create an account to join the conversation.

drbd continuously stalling and slow on initial sync 3 years 4 months ago #2254

*headbang*
Ok, finally I know the reason for the slowness - the technician put the two super micro servers with gigabit nics for fast network response on an really old 100mbit switch :-O= .... Now timings are much better. Although the dropped packets didn't disappear, speed is much faster now ;-)) As expected 10x as fast....

version: 8.4.10 (api:1/proto:86-101)
srcversion: CE41F2FAB70AE3534BF40A6

1: cs:SyncTarget ro:Primary/Secondary ds:Inconsistent/UpToDate C r
ns:1306633812 nr:984633096 dw:143795244 dr:140090332 al:282347 bm:0 lo:9 pe
:21 ua:8 ap:0 ep:1 wo:f oos:8043726040
[>....................] sync'ed: 2.0% (7855200/8009468)M
finish: 22:48:49 speed: 97,920 (91,896) want: 102,400 K/sec

But now I am trying to speed it up for 1gbit link speed. this is dedicated, no need for slowness.

Please Log in or Create an account to join the conversation.

  • Page:
  • 1